update rustdoc's syntax checker to work with error-recovering lexer

[matklad]

Rustdoc processes code inside of ``` blocks in two ways:

• First, the code is checked for lexer errors

• Second, lexer-based syntax highlighting is done

The checking for lexer errors works by intercepting fatal errors from the lexer. However, since this was originally implemented, the lexer moved from fatal erroring to error-recovery (#63017 in particular tries to remove the last bit of fatal erroring). That means that the current approach intercepts only a fraction of lexer errors, while most of the errors are reported twice (once duing the check, once during highlighing). Here's an example:

14:30:41|~/tmp
λ cat main.rs 
/// ```
/// '...'
/// ```
pub fn foo() {}

14:31:11|~/tmp
λ rustdoc main.rs
error: character literal may only contain one codepoint
 --> <doctest>:1:1
  |
1 | '...'
  | ^^^^^
help: if you meant to write a `str` literal, use double quotes
  |
1 | "..."
  |

error: character literal may only contain one codepoint
 --> <rustdoc-highlighting>:1:1
  |
1 | '...'
  | ^^^^^
help: if you meant to write a `str` literal, use double quotes
  |
1 | "..."

I think that, to fix this, we should configure the parsing session with a custom Emitter. For code-checking pass, the emmitter should downgrade all diagnostics to warnings and set a flag if there were any diagnostics. For syntax-highlighting pass, we should use a "/dev/null" emitter which just doesn't emit anything.

cc @euclio, @GuillaumeGomez

[euclio]

cc #56885

[matklad]

hm, #56885 makes me think that there are two ways to approach this:

• use special emitter (what issue description is proposing)
• make all lexer errors buffered

I don't know what would be better (someone with more broad compiler knowledge needs to decide this), but I like BufferingEmitter approach better, because it can be reused anywhere, while buffering in the lexer will be special-casing the lexer.

[estebank]

An alternative would be to have the lexer and parser have a "strict" mode where they bail early when encountering any error, that way the caller can handle the Err case themselves. This would also help people using the Parser as a library.

[Centril]

@estebank That sounds like it would complicate parser.rs (which is already in a sorry state!).

[Mark-Simulacrum]

A separate/custom emitter is definitely the way to go here, we should not complicate parsing code anymore than it already is.

[jyn514]

@matklad this only gives one warning on master, I think it was fixed by #76068 ?

[camelid]

Should this be closed?