sync: handle docs, metadata, and filepaths

[ee7]

To-do:

• Support parsing new options: --docs, --filepaths, --metadata, --tests, -y
• Rename check.nim to sync.nim, and update.nim to update_tests.nim
• Clean up update_tests.nim
• Add initial docs checking
• Add initial docs updating
• Add initial metadata checking
• Add initial metadata updating
• Add initial filepaths checking
• Add initial filepaths updating
• Merge the open PRs and rebase this PR on top
• (Now on main) test --exercise
• (Now on main) test --update with y, n, and something else from stdin
• Consider a separate PR for ff19da4 to help git track the renames. Currently if we squash this PR, git probably doesn't know that sync.nim comes from check.nim. Edit: Let's not bother.
• Change help message of --exercise
• Change help message of --yes to clarify that it only affects docs, metadata, and filepaths
• Change help message of --update
• Test -y
• filepaths: don't clone prob-specs when no other scope
• Make sync -uy and sync -uy --tests produce an error
• Check the behavior in the edge cases like: exercise config.json doesn't exist, lacks a files key, etc.
• filepaths, metadata: use only jsony for parsing JSON
• filepaths, metadata: deserialize without std/json
• Fix various messages
• userSaysYes: handle unrecognised entry
• Only prompt when stderr is a tty
• jsony: improve error messages, pointing to line + char of JSON parsing error
• Just include the fix for #31 in this PR
• docs, filepaths, metadata: improve type safety: enforce that a Slug is kebab-case.
• docs: fix needlessly slow file comparison
• docs: resolve the TODO item regarding the top-level header
• docs: create e.g. instructions.md when it doesn't exist
• Do a quick profiling pass
• Optimize away joinPath allocations
• Improve log readability
• (Omitting a large number of other, mostly smaller, completed TODO items)
• tests: json serialization: Test basic serialization and that we serialize every .meta/config.json file on a track the same as parseJson.toPretty() with some keys removed
• Fix the Windows build
• Don't create a missing directory until the user confirms
• Tweak --mode choose|include|exclude into --tests choose|include|exclude
• Go back to pkg/parsetoml for metadata parsing
• --filepaths, --metadata: support exercise config containing custom key with the value of any valid JSON object
• Patch parsing of --tests --foo
• Fix bug with "custom": {}
• Add test for problematic CLI parsing case
• Make configlet sync --filepaths -u and configlet sync --metadata -u preserve key order
• Reply to Erik's comments
• Remove fmt code

Final pass:

• Ensure help message in README.md is up-to-date with the actual configlet -h
• Update README.md to describe the new sync behavior
• Run nimpretty
• Change any new proc to func where possible
• Only export when necessary
• Check that doc comments are still correct
• Check the output of configlet sync on every track

Fixes: #31
Closes: #298

---

After merging:

• Tweak key order for configlet fmt?
• Add configlet fmt
• Further improve the text in the configlet --help message (I have an unpushed commit that adds a description to every command in the help message).
• Fix message for missing prob-specs dir (#74)
• Create a release
• Update exercism/problem-specifications README.md
• Update configlet docs in exercism/docs

Then things like:

• Pin dependency versions
• Cache prob-specs repo
• Test --update --docs
• Test --update --filepaths
• Test --update --metadata
• Improve --tests -uy message/interaction further?
• Maybe make everything an Option[foo] - do we want to have an actual JSON parsing error for things like contributors: null, or e.g. silently replace it on updating?
• tests: Split the tests of the configlet executable into separate files
• tests: refactor those tests
• Maybe don't say "filepaths are up to date". Maybe something like "filepaths are already populated?"
• --update --exercise foo: improve message consistency
• --update --yes: put updated summary at the end?
• --filepaths: warn for invalid placeholders
• Improve logging and error writing
• colors: When stdout is a tty, consider [skip] green; [warn] yellow; [error] red
• Add more messages in verbose mode
• Refactor to use common write proc
• Print number of unsynced exercises
• Hint/warn for e.g configlet sync --metadata -e lasagna (can't sync docs/filepaths/tests for a Concept Exercise)
• Make --tests use jsony only
• tests: Test behavior when there are extra key/value pairs
• Try to refactor some var parameters
• Remove some cruft from the tests after updating tests.toml in Nim track repo and bumping the Nim commit ref

[ErikSchierboom]

Thanks for working on this! I've not done any testing yet, as I think you are still working on this a bit. Let me know when you want me to test.

[ee7]

@ErikSchierboom If an exercise lacks a config.json file, what should these commands do?

• configlet sync --filepaths
• configlet sync --filepaths --update
• configlet sync --metadata
• configlet sync --metadata --update

For the --update ones, we want to create the config.json if valid values exist at the upstream source, rather than just complaining that the file doesn't exist?

(Similar to exercism/docs#155 (comment))

[ErikSchierboom]

For the --update ones, we want to create the config.json if valid values exist at the upstream source, rather than just complaining that the file doesn't exist?

I think that would be nice yeah. It's not a hard requirement, but it would make sense to me.

For the non-update ones, maybe output an error?

[ee7]

I'll update this PR with the latest changes from main soon.

[ErikSchierboom]

🥳 Awesome work!