Split up `Definitions` and `ResolverAstLowering`. #98106

cjgillot · 2022-06-14T19:43:33Z

Split off #95573

michaelwoerister

Thanks, @cjgillot! Looks good to me.

To clarify, with this change we lose the ability to associated source_spans with items created during lowering, right? Since the PR did not have to update any tests, it looks like that doesn't lead to regression in diagnostics quality, so that should be fine.

r=me with the comment below addressed.

michaelwoerister · 2022-06-15T08:30:43Z

compiler/rustc_ast_lowering/src/lib.rs

@@ -133,6 +133,10 @@ struct LoweringContext<'a, 'hir: 'a> {
    /// NodeIds that are lowered inside the current HIR owner.
    node_id_to_local_id: FxHashMap<NodeId, hir::ItemLocalId>,

+    // The next_node_id is reset for each item.


I think it would be good to elaborate this comment. It's hard to tell what it means in the current form. Something like:

Note: We create a new LoweringContext for each item, and each time we initialize next_node_id with resolver.next_node_id, which means that each time we start with the same value and the id ranges for new nodes created during lowering will overlap. This is fine because these newly created NodeIds don't escape the item's context and are immediately lowered into HirIds using the item-local node_id_to_local_id map.

petrochenkov · 2022-06-15T13:26:05Z

compiler/rustc_ast_lowering/src/lib.rs

+    fn next_node_id(&mut self) -> NodeId {
+        let next = self
+            .next_node_id
+            .as_usize()


Looks like this version didn't incorporate some recent changes in fn next_node_id.

petrochenkov · 2022-06-15T13:43:16Z

compiler/rustc_middle/src/ty/mod.rs

+    pub trait_map: NodeMap<Vec<hir::TraitCandidate>>,
+    /// A small map keeping true kinds of built-in macros that appear to be fn-like on
+    /// the surface (`macro` items in libcore), but are actually attributes or derives.
+    pub builtin_macro_kinds: FxHashMap<LocalDefId, MacroKind>,


I remember these changes from #90451.

The comments from #90451 (comment) are still relevant.
It doesn't seem right that all this data is only used during lowering, but then lives unused (or potentially accidentally used in the future) as long as tcx.

It should be better to move this data to a separate structure.
Then AST lowerer should be able to consume it by value.

petrochenkov · 2022-06-15T21:06:09Z

compiler/rustc_resolve/src/lib.rs

-    pub fn into_outputs(self) -> ResolverOutputs {
+    pub fn into_outputs(
+        self,
+    ) -> (Definitions, Box<CrateStoreDyn>, ResolverOutputs, ty::ResolverAstLowering) {


This tuple got so large that it makes sense to turn it into a structure, named ResolverOutputs for example 😄
The old ResolverOutputs could get some more specific name in that case.

michaelwoerister · 2022-06-16T08:40:49Z

I like the updated version. It looks like they address @petrochenkov's comment.

Let's do a perf run. But overall, I think this is ready to be merged (unless @petrochenkov has further comments).
@bors try @rust-timer queue

rust-timer · 2022-06-16T08:40:51Z

Awaiting bors try build completion.

@rustbot label: +S-waiting-on-perf

bors · 2022-06-16T08:40:58Z

⌛ Trying commit ae5959f with merge 1270191ac8f3e413cacc6ad5e929641b86774c3b...

bors · 2022-06-16T10:31:44Z

☀️ Try build successful - checks-actions
Build commit: 1270191ac8f3e413cacc6ad5e929641b86774c3b (1270191ac8f3e413cacc6ad5e929641b86774c3b)

rust-timer · 2022-06-16T10:31:45Z

Queued 1270191ac8f3e413cacc6ad5e929641b86774c3b with parent 392d272, future comparison URL.

rust-timer · 2022-06-16T12:36:56Z

Finished benchmarking commit (1270191ac8f3e413cacc6ad5e929641b86774c3b): comparison url.

Instruction count

Primary benchmarks: 🎉 relevant improvements found
Secondary benchmarks: mixed results

	mean¹	max	count²
Regressions 😿 (primary)	N/A	N/A	0
Regressions 😿 (secondary)	0.5%	0.5%	2
Improvements 🎉 (primary)	-0.3%	-0.6%	32
Improvements 🎉 (secondary)	-0.4%	-1.0%	6
All 😿🎉 (primary)	-0.3%	-0.6%	32

Max RSS (memory usage)

Results

Primary benchmarks: 🎉 relevant improvements found
Secondary benchmarks: mixed results

	mean¹	max	count²
Regressions 😿 (primary)	1.0%	1.0%	2
Regressions 😿 (secondary)	3.2%	4.3%	5
Improvements 🎉 (primary)	-1.7%	-4.3%	14
Improvements 🎉 (secondary)	-5.1%	-8.1%	17
All 😿🎉 (primary)	-1.3%	-4.3%	16

Cycles

Results

Primary benchmarks: no relevant changes found
Secondary benchmarks: mixed results

	mean¹	max	count²
Regressions 😿 (primary)	N/A	N/A	0
Regressions 😿 (secondary)	3.3%	3.3%	1
Improvements 🎉 (primary)	N/A	N/A	0
Improvements 🎉 (secondary)	-2.8%	-2.8%	1
All 😿🎉 (primary)	N/A	N/A	0

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: +S-waiting-on-review -S-waiting-on-perf -perf-regression

the arithmetic mean of the percent change ↩ ↩² ↩³
number of relevant changes ↩ ↩² ↩³

michaelwoerister · 2022-06-17T07:19:50Z

OK, performance looks good.
@bors r+

bors · 2022-06-17T07:19:52Z

📌 Commit ae5959f has been approved by michaelwoerister

bors · 2022-06-17T10:00:14Z

⌛ Testing commit ae5959f with merge 3a8b014...

bors · 2022-06-17T12:41:18Z

☀️ Test successful - checks-actions
Approved by: michaelwoerister
Pushing 3a8b014 to master...

rust-timer · 2022-06-17T14:13:45Z

Finished benchmarking commit (3a8b014): comparison url.

Instruction count

Primary benchmarks: 🎉 relevant improvements found
Secondary benchmarks: mixed results

	mean¹	max	count²
Regressions 😿 (primary)	N/A	N/A	0
Regressions 😿 (secondary)	0.4%	0.5%	9
Improvements 🎉 (primary)	-0.3%	-0.6%	39
Improvements 🎉 (secondary)	-0.3%	-0.6%	5
All 😿🎉 (primary)	-0.3%	-0.6%	39

Max RSS (memory usage)

Results

Primary benchmarks: 🎉 relevant improvements found
Secondary benchmarks: mixed results

	mean¹	max	count²
Regressions 😿 (primary)	N/A	N/A	0
Regressions 😿 (secondary)	2.5%	3.8%	5
Improvements 🎉 (primary)	-1.7%	-4.2%	14
Improvements 🎉 (secondary)	-4.7%	-8.1%	19
All 😿🎉 (primary)	-1.7%	-4.2%	14

Cycles

Results

Primary benchmarks: no relevant changes found
Secondary benchmarks: 🎉 relevant improvement found

	mean¹	max	count²
Regressions 😿 (primary)	N/A	N/A	0
Regressions 😿 (secondary)	N/A	N/A	0
Improvements 🎉 (primary)	N/A	N/A	0
Improvements 🎉 (secondary)	-2.7%	-2.7%	1
All 😿🎉 (primary)	N/A	N/A	0

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

@rustbot label: -perf-regression

the arithmetic mean of the percent change ↩ ↩² ↩³
number of relevant changes ↩ ↩² ↩³

rustc_middle: Rearrange resolver outputs structures slightly Addresses rust-lang#98106 (comment). I also haven't seen the motivation for moving `cstore` from its old place, so I moved it back in this PR. r? `@cjgillot`

rustc_middle: Rearrange resolver outputs structures slightly Addresses rust-lang#98106 (comment). I also haven't seen the motivation for moving `cstore` from its old place, so I moved it back in this PR. r? ``@cjgillot``

rustc_middle: Rearrange resolver outputs structures slightly Addresses rust-lang#98106 (comment). I also haven't seen the motivation for moving `cstore` from its old place, so I moved it back in this PR. r? ```@cjgillot```

rust-highfive assigned michaelwoerister Jun 14, 2022

rustbot added T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. labels Jun 14, 2022

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 14, 2022

petrochenkov self-assigned this Jun 14, 2022

This comment was marked as resolved.

Sign in to view

cjgillot added 4 commits June 14, 2022 22:44

Separate Definitions and CrateStore from ResolverOutputs.

47799de

Make ResolverAstLowering a struct.

603746a

Do not modify the resolver outputs.

b676edd

Separate source_span and expn_that_defined from Definitions.

34e4d72

cjgillot force-pushed the split-definitions branch from 5c1245c to 34e4d72 Compare June 14, 2022 20:46

michaelwoerister reviewed Jun 15, 2022

View reviewed changes

petrochenkov reviewed Jun 15, 2022

View reviewed changes

petrochenkov added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 15, 2022

cjgillot added 3 commits June 15, 2022 19:19

Remove unused item_generics_num_lifetimes.

52f22c7

Update next_node_id.

8ffbd81

Consume resolutions for lowering separately.

ae5959f

petrochenkov removed their assignment Jun 15, 2022

petrochenkov reviewed Jun 15, 2022

View reviewed changes

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jun 16, 2022

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Jun 16, 2022

bors added the merged-by-bors This PR was explicitly merged by bors. label Jun 17, 2022

bors merged commit 3a8b014 into rust-lang:master Jun 17, 2022

rustbot added this to the 1.63.0 milestone Jun 17, 2022

bors mentioned this pull request Jun 17, 2022

fix: impl Trait desugaring in trait objects' assoc constraints #97335

Closed

8 tasks

cjgillot deleted the split-definitions branch June 17, 2022 20:20

JohnTitor mentioned this pull request Jul 26, 2022

Fix some links rust-lang/rustc-dev-guide#1411

Merged

petrochenkov mentioned this pull request Oct 25, 2022

rustc_middle: Rearrange resolver outputs structures slightly #103520

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Split up `Definitions` and `ResolverAstLowering`. #98106

Split up `Definitions` and `ResolverAstLowering`. #98106

cjgillot commented Jun 14, 2022

This comment was marked as resolved.

michaelwoerister left a comment

michaelwoerister Jun 15, 2022

petrochenkov Jun 15, 2022

petrochenkov Jun 15, 2022

petrochenkov Jun 15, 2022

michaelwoerister commented Jun 16, 2022

rust-timer commented Jun 16, 2022

bors commented Jun 16, 2022

bors commented Jun 16, 2022

rust-timer commented Jun 16, 2022

rust-timer commented Jun 16, 2022

michaelwoerister commented Jun 17, 2022

bors commented Jun 17, 2022

bors commented Jun 17, 2022

bors commented Jun 17, 2022

rust-timer commented Jun 17, 2022

Split up Definitions and ResolverAstLowering. #98106

Split up Definitions and ResolverAstLowering. #98106

Conversation

cjgillot commented Jun 14, 2022

This comment was marked as resolved.

michaelwoerister left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

michaelwoerister commented Jun 16, 2022

rust-timer commented Jun 16, 2022

bors commented Jun 16, 2022

bors commented Jun 16, 2022

rust-timer commented Jun 16, 2022

rust-timer commented Jun 16, 2022

Footnotes

michaelwoerister commented Jun 17, 2022

bors commented Jun 17, 2022

bors commented Jun 17, 2022

bors commented Jun 17, 2022

rust-timer commented Jun 17, 2022

Footnotes

Split up `Definitions` and `ResolverAstLowering`. #98106

Split up `Definitions` and `ResolverAstLowering`. #98106