Fix parenthesization of subexprs containing statement boundary

[dtolnay]

This PR fixes a multitude of false negatives and false positives in the AST pretty printer's parenthesis insertion related to statement boundaries — statements which terminate unexpectedly early if there aren't parentheses.

Without this fix, the AST pretty printer (including both stringify! and rustc -Zunpretty=expanded) is prone to producing output which is not syntactically valid Rust. Invalid output is problematic because it means Rustfmt is unable to parse the output of cargo expand, for example, causing friction by forcing someone trying to debug a macro into reading poorly formatted code.

I believe the set of bugs fixed in this PR account for the most prevalent reason that cargo expand produces invalid output in real-world usage.

Fixes #98790.

False negatives

The following is a correct program — cargo check succeeds.

macro_rules! m {
    ($e:expr) => {
        match () { _ => $e }
    };
}

fn main() {
    m!({ 1 } - 1);
}

But rustc -Zunpretty=expanded main.rs produces output that is invalid Rust syntax, because parenthesization is needed and not being done by the pretty printer.

fn main() { match () { _ => { 1 } - 1, }; }

Piping this expanded code to rustfmt, it fails to parse.

error: unexpected `,` in pattern
 --> <stdin>:1:38
  |
1 | fn main() { match () { _ => { 1 } - 1, }; }
  |                                      ^
  |
help: try adding parentheses to match on a tuple...
  |
1 | fn main() { match () { _ => { 1 } (- 1,) }; }
  |                                   +    +
help: ...or a vertical bar to match on multiple alternatives
  |
1 | fn main() { match () { _ => { 1 } - 1 | }; }
  |                                   ~~~~~

Fixed output after this PR:

fn main() { match () { _ => ({ 1 }) - 1, }; }

False positives

Less problematic, but worth fixing (just like #118726).

fn main() {
    let _ = match () { _ => 1 } - 1;
}

Output of rustc -Zunpretty=expanded lib.rs before this PR. There is no reason parentheses would need to be inserted there.

fn main() { let _ = (match () { _ => 1, }) - 1; }

After this PR:

fn main() { let _ = match () { _ => 1, } - 1; }

Alternatives considered

In this PR I opted to parenthesize only the leading subexpression causing the statement boundary, rather than the entire statement. Example:

macro_rules! m {
    ($e:expr) => {
        $e
    };
}

fn main() {
    m!(loop { break [1]; }[0] - 1);
}

This PR produces the following pretty-printed contents for fn main:

(loop { break [1]; })[0] - 1;

A different equally correct output would be:

(loop { break [1]; }[0] - 1);

I chose the one I did because it is the only approach used by handwritten code in the standard library and compiler. There are 4 places where parenthesization is being used to prevent a statement boundary, and in all 4, the developer has chosen to parenthesize the smallest subexpression rather than the whole statement:

rust/compiler/rustc_codegen_cranelift/example/alloc_system.rs

Line 102 in b37d43e

(if ptr.is_null() { ptr } else { align_ptr(ptr, layout.align()) }) as *mut u8

rust/compiler/rustc_parse/src/errors.rs

Lines 1021 to 1029 in b37d43e

	(match token_descr {
	Some(TokenDescription::ReservedIdentifier) => {
	ExpectedIdentifierFound::ReservedIdentifier
	}
	Some(TokenDescription::Keyword) => ExpectedIdentifierFound::Keyword,
	Some(TokenDescription::ReservedKeyword) => ExpectedIdentifierFound::ReservedKeyword,
	Some(TokenDescription::DocComment) => ExpectedIdentifierFound::DocComment,
	None => ExpectedIdentifierFound::Other,
	})(span)

rust/library/core/src/future/poll_fn.rs

Line 151 in b37d43e

(unsafe { &mut self.get_unchecked_mut().f })(cx)

rust/library/core/src/ops/range.rs

Lines 824 to 828 in b37d43e

	(match self.start_bound() {
	Included(start) => start <= item,
	Excluded(start) => start < item,
	Unbounded => true,
	}) && (match self.end_bound() {

[rustbot]

r? @WaffleLapkin

(rustbot has picked a reviewer for you, use r? to override)

[WaffleLapkin]

I'm surprised that stringify is expected to produce valid rust, as opposed to "what's literally written in the file", I but given it can observe macro expansion and there is precedence for this, I'm inclined to say that it's fine.

Will review the code a bit later.

[dtolnay]

For stringify!, I agree it is not obvious that we need it to produce valid Rust syntax. I do not know of any use cases that involve parsing the output of stringify! other than visually by a human (for which valid syntax is nice but not a hard requirement).

But there is only one AST pretty-printer, and stringify! and rustc -Zunpretty=expanded use the same one. I care 100× more about the latter producing valid syntax, because cargo expand is based on it and relies on parsing the output not just for piping to rustfmt / prettyplease, but also for cargo expand path::to::module where it filters the output down to just a single item by path, which I use commonly.

[WaffleLapkin]

@bors r+

[bors]

📌 Commit 17239d9 has been approved by WaffleLapkin

It is now in the queue for this repository.

[bors]

⌛ Testing commit 17239d9 with merge 89e2160...

[bors]

☀️ Test successful - checks-actions
Approved by: WaffleLapkin
Pushing 89e2160 to master...

[rust-timer]

Finished benchmarking commit (89e2160): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	1.0%	[1.0%, 1.0%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.1%	[-0.1%, -0.1%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.4%	[-0.1%, 1.0%]	2

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.6%	[0.6%, 0.6%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.6%	[0.6%, 0.6%]	1

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 672.366s -> 670.933s (-0.21%)
Artifact size: 312.33 MiB -> 312.35 MiB (0.00%)