Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compute lint_levels by definition #99634

Conversation

fee1-dead
Copy link
Member

Closes #95094.

cc @cjgillot

@rustbot rustbot added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Jul 23, 2022
@rust-highfive
Copy link
Collaborator

r? @nagisa

(rust-highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jul 23, 2022
@fee1-dead
Copy link
Member Author

cc @xFrednet this touches impl for lint expectations

@nagisa
Copy link
Member

nagisa commented Jul 23, 2022

r? rust-lang/compiler

Not able to take on large reviews for the time being.

@rust-highfive rust-highfive assigned davidtwco and unassigned nagisa Jul 23, 2022
@cjgillot
Copy link
Contributor

@bors try @rust-timer queue

@rust-timer
Copy link
Collaborator

Awaiting bors try build completion.

@rustbot label: +S-waiting-on-perf

@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jul 24, 2022
@bors
Copy link
Contributor

bors commented Jul 24, 2022

⌛ Trying commit 224cfded3fde57789375420501eb2146971eb807 with merge f2b76fa8b8a81c9059018568c196cf62f6ba4fdf...

@bors
Copy link
Contributor

bors commented Jul 24, 2022

☀️ Try build successful - checks-actions
Build commit: f2b76fa8b8a81c9059018568c196cf62f6ba4fdf (f2b76fa8b8a81c9059018568c196cf62f6ba4fdf)

@rust-timer
Copy link
Collaborator

Queued f2b76fa8b8a81c9059018568c196cf62f6ba4fdf with parent c32dcbb, future comparison URL.

@rust-timer
Copy link
Collaborator

Finished benchmarking commit (f2b76fa8b8a81c9059018568c196cf62f6ba4fdf): comparison url.

Instruction count

  • Primary benchmarks: mixed results
  • Secondary benchmarks: mixed results
mean1 max count2
Regressions 😿
(primary)
14.5% 62.7% 94
Regressions 😿
(secondary)
23.8% 254.5% 75
Improvements 🎉
(primary)
-0.6% -1.2% 12
Improvements 🎉
(secondary)
-0.7% -1.3% 37
All 😿🎉 (primary) 12.8% 62.7% 106

Max RSS (memory usage)

Results
  • Primary benchmarks: 😿 relevant regressions found
  • Secondary benchmarks: 😿 relevant regressions found
mean1 max count2
Regressions 😿
(primary)
10.6% 26.3% 77
Regressions 😿
(secondary)
6.9% 16.6% 31
Improvements 🎉
(primary)
N/A N/A 0
Improvements 🎉
(secondary)
N/A N/A 0
All 😿🎉 (primary) 10.6% 26.3% 77

Cycles

Results
  • Primary benchmarks: 😿 relevant regressions found
  • Secondary benchmarks: 😿 relevant regressions found
mean1 max count2
Regressions 😿
(primary)
25.1% 86.5% 78
Regressions 😿
(secondary)
37.8% 259.4% 51
Improvements 🎉
(primary)
-2.5% -2.8% 2
Improvements 🎉
(secondary)
N/A N/A 0
All 😿🎉 (primary) 24.4% 86.5% 80

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: +S-waiting-on-review -S-waiting-on-perf +perf-regression

Footnotes

  1. the arithmetic mean of the percent change 2 3

  2. number of relevant changes 2 3

@rustbot rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Jul 24, 2022
Copy link
Member

@davidtwco davidtwco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not exceptionally familiar with the lint level computation but this seems reasonable (when not considering the big perf regression), will re-assign to cjgillot though.

@davidtwco
Copy link
Member

r? @cjgillot

@rust-highfive rust-highfive assigned cjgillot and unassigned davidtwco Jul 25, 2022
@fee1-dead
Copy link
Member Author

fee1-dead commented Jul 27, 2022

Yeah, I'm going to need some help regarding the perf hits here. @cjgillot could you take a look? Thanks.

Copy link
Contributor

@cjgillot cjgillot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You caught a bad case of incremental compilation invalidation.

When the incr. comp. engine tries to mark a query as green, it first checks whether all its dependencies are green. lint_levels_on depends on resolutions, which is always red.
As it can trivially mark lint_levels_on as green, the incr. comp. engine tries to recompute the query. The query takes a HirId but the incr. comp. engine does not know how to make a HirId from its Fingerprint. So it cannot recompute the query, and marks lint_levels_on as red.
As a consequence, all queries that call it (= pretty much everything) get recomputed.

There are 2 possibilities here:

  1. compute lint_levels_on by definition: for a whole item/trait-item/impl-item/foreign-item at once, and make lint_levels_on take a LocalDefId;
  2. teach the query system to force a query that takes a HirId.

(1) may need to a larger refactor of the levels computation. It will avoid the query call overhead (there are many more HirIds that LocalDefIds), but that overhead only appears as a ~1% regression, so that's probably fine.

(2) is a bit more experimental. For this, you need to:

  • add a variant to rustc_query_system::dep_graph::FingerprintStyle;
  • return it in the <HirId as DepNodeParams>::fingerprint_style in rustc_middle::dep_graph::dep_node;
  • modify to_fingerprint and recover in that method have a bidirectional conversion Fingerprint <-> HirId (look at what LocalDefId does for instance).

@fee1-dead fee1-dead force-pushed the compute_lint_levels_by_def branch from 224cfde to 4e32372 Compare July 30, 2022 18:01
@fee1-dead
Copy link
Member Author

rebase; no work yet

@cjgillot cjgillot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 7, 2022
@bors
Copy link
Contributor

bors commented Aug 12, 2022

☔ The latest upstream changes (presumably #100426) made this pull request unmergeable. Please resolve the merge conflicts.

@fee1-dead fee1-dead force-pushed the compute_lint_levels_by_def branch from 4e32372 to 0899c7c Compare August 17, 2022 15:05
@rust-log-analyzer
Copy link
Collaborator

The job mingw-check failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)
configure: rust.debug-assertions := True
configure: rust.overflow-checks := True
configure: llvm.assertions      := True
configure: dist.missing-tools   := True
configure: build.configure-args := ['--enable-sccache', '--disable-manage-submodu ...
configure: writing `config.toml` in current directory
configure: 
configure: run `python /checkout/x.py --help`
Attempting with retry: make prepare
---
skip untracked path cpu-usage.csv during rustfmt invocations
skip untracked path src/doc/book/ during rustfmt invocations
skip untracked path src/doc/rust-by-example/ during rustfmt invocations
skip untracked path src/llvm-project/ during rustfmt invocations
Diff in /checkout/compiler/rustc_lint/src/levels.rs at line 187:
         intravisit::walk_field_def(self, s);
 
-    fn visit_variant(
-        &mut self,
-        &mut self,
-        v: &'tcx hir::Variant<'tcx>,
-    ) {
+    fn visit_variant(&mut self, v: &'tcx hir::Variant<'tcx>) {
         self.add_id(v.id);
         intravisit::walk_variant(self, v);
     }
Running `"/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/rustfmt" "--config-path" "/checkout" "--edition" "2021" "--unstable-features" "--skip-children" "--check" "/checkout/compiler/rustc_lint/src/nonstandard_style.rs" "/checkout/compiler/rustc_lint/src/levels.rs" "/checkout/compiler/rustc_lint/src/types.rs" "/checkout/compiler/rustc_lint/src/enum_intrinsics_non_enums.rs" "/checkout/compiler/rustc_lint/src/context.rs" "/checkout/compiler/rustc_lint/src/hidden_unicode_codepoints.rs" "/checkout/compiler/rustc_lint/src/late.rs" "/checkout/compiler/rustc_mir_build/src/thir/cx/block.rs"` failed.
If you're running `tidy`, try again with `--bless`. Or, if you just want to format code, run `./x.py fmt` instead.

@bors
Copy link
Contributor

bors commented Aug 27, 2022

☔ The latest upstream changes (presumably #101064) made this pull request unmergeable. Please resolve the merge conflicts.

@fee1-dead
Copy link
Member Author

Closing in favor of #101620. Thank you for working on this @cjgillot!

@fee1-dead fee1-dead closed this Sep 10, 2022
bors added a commit to rust-lang-ci/rust that referenced this pull request Sep 15, 2022
… r=oli-obk

Compute lint levels by definition

Lint levels are currently computed once for the whole crate. Any code that wants to emit a lint depends on this single `lint_levels(())` query. This query contains the `Span` for each attribute that participates in the lint level tree, so any code that wants to emit a lint basically depends on the spans in all files in the crate.

Contrary to hard errors, we do not clear the incremental session on lints, so this implicit world dependency pessimizes incremental reuse. (And is furthermore invisible for allowed lints.)

This PR completes rust-lang#99634 (thanks for the initial work `@fee1-dead)` and includes it in the dependency graph.

The design is based on 2 queries:
1. `lint_levels_on(HirId) -> FxHashMap<LintId, LevelAndSource>` which accesses the attributes at the given `HirId` and processes them into lint levels.  The `TyCtxt` is responsible for probing the HIR tree to find the user-visible level.
2. `lint_expectations(())` which lists all the `#[expect]` attributes in the crate.

This PR also introduces the ability to reconstruct a `HirId` from a `DepNode` by encoding the local part of the `DefPathHash` and the `ItemLocalId` in the two `u64` of the fingerprint.  This allows for the dep-graph to directly recompute `lint_levels_on` directly, without having to force the calling query.

Closes rust-lang#95094.
Supersedes rust-lang#99634.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
perf-regression Performance regression. S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Experiment computing lint_levels by definition
9 participants