Proposal: Add Markdown related jobs to flowzone

[vipulgupta2048]

Would it be a good idea to add markdown related flows to flowzone?

The docs runs a spellchecker using a tool called Cspell and a link checker called lychee.
IMHO these checks could be really helpful for repos like handbook, balena-site, masterclasses or any product repo holding markdown docs.

Usecases:

• Atm, if masterclasses have a spelling mistake then it only gets notified when they are pulled into the docs and fails a random PR. Having cspell on masterclasses could have caught it.
• Docs have their own cspell dictionary as well containing many balena words thanks to @zoobot, we can generalize this for the entire company.

[zoobot]

Good idea, then we don't have to deal with upstream misspellings. Wouldn't want custom dictionary words to be a blocker for PRs across all the repos though so need an idea around that. cspell has an add word feature, not sure if that's feasible to have as part of the CI to add words on the fly during tests. @vipulgupta2048
I need to give flowzone a once over to see what it requires to add tests like this.

[vipulgupta2048]

Who do we reach out to have a discussion on this first? I was hoping to get more comments in first before rushing to execute it. I know it will be an improvement from what we currently have but let's get this done right.

Spellchecker Questions

• Do we let cspell spellcheck the code? or just the docs of a product? Are there any specific files/folders we need to be looking into?
• We can create a central balena dictionary in flowzone for it but what about custom product-related dictionaries? Moreover, how do we define the central balena dictionary? (We can definitely start with the base from the docs repo)
• Which repos will get cspell as a required check?
• We need to enforce a central config for spellcheck for all repos? What would that look like? How to custom configs?
• More questions...

Linkchecker questions

• Lychee can find broken links in any text file. Which files/directories should we enable it for?
• In docs took quite a bit of config tweaks for it to scan all 16k+ links in the docs effectively. How do we do custom configs? How are multiple configs handled by lychee
• More questions?

[zoobot]

Adding this here as examples of remote dictionaries.
@Bucknalla - "we can point directly at a dictionary repo then, I much prefer that for maintenance"

"ignoreRegExpList": [ "/https%3A%2F%2F.*?/gi" ]
"dictionaryDefinitions": [
{
"name": "balena",
"path": "./.cspell/balena-words.txt",
"addWords": true
},
{
"name": "computerTerms",
// "path": "./.cspell/computer-terms.txt",
"path": "https://raw.githubusercontent.com/balena-io/docs/cspell-fix/.cspell/balena-words.txt",
"addWords": true
}
],
"exclude": ["/.svg"],
"ignorePaths": ["node_modules",
"CHANGELOG.md",
"https://raw.githubusercontent.com/balenablocks/local-ui/master/README.md"
]

[zoobot]

Should we trial cspell just in docs for a month or so before flowzone roll out?

[vipulgupta2048]

SGTM, I might even bring this up in arch call? The question of having Markdown related checks even worth or not is probably worth discussing

[vipulgupta2048]

I think that's what we are doing.