Validate test / code-block configuration

[mauvealerts]

It seems like some combinations of code-block tags are non-sensical. I've thought of:

• ignore with compile_fail, no_run, or should_panic
• compile_fail with no_run or should_panic
• no_run with should_panic

It'd be nice to detect these surprising combinations. Offhand, I see two options:

• Fail when parsing, but there's no way to handle an error per se (at a glance?)
• During handle_test, returning TestResult::Invalid (or something like that)

My intuition is to that the handle_test is cleaner. However, I think we'd need to generalize a bit from Output or rework the control flow in print_results or something like that.

[mgeisler]

Cool that you're looking at this! 🙂

It'd be nice to detect these surprising combinations.

Would it make sense to align with what mdbook test does (or doesn't do) here?

I think mdbook-keeper doesn't aim for compatibility and so it can decide to be stricter. Compatibility would mean something like "if mdbook test passes, then mdbook-keeper will also pass".

[mauvealerts]

Based on a skim of librustdoc (used by mdbook test) and some testing, it seems that there's no validation of code block options. Instead, there's some implicit precedence. I also did a quick search for RFCs and T-rustdoc issues. The closest thing I found is rust-lang/rust#71347 (Emit a warning when a codeblock is using "compile-fail" instead of "compile_fail"). For lack of a clearer place to start, I'll try making a topic in the rustdoc Zulip stream (later).

[tfpk]

Thanks for looking into this -- let me know if there's anything I can do to help.