-
Notifications
You must be signed in to change notification settings - Fork 12.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make create_def a side effect instead of marking the entire query as always red #115613
base: master
Are you sure you want to change the base?
Changes from all commits
483934c
52f148f
4ef3160
26edb9d
5ebfe8a
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -1113,7 +1113,6 @@ rustc_queries! { | |
"evaluating initializer of static `{}`", | ||
tcx.def_path_str(key) | ||
} | ||
cache_on_disk_if { key.is_local() } | ||
separate_provide_extern | ||
feedable | ||
} | ||
|
@@ -1841,6 +1840,7 @@ rustc_queries! { | |
desc { |tcx| "computing visibility of `{}`", tcx.def_path_str(def_id) } | ||
separate_provide_extern | ||
feedable | ||
cache_on_disk_if { def_id.is_local() } | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should this be benchmarked and landed separately? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oh whoops, this is not a perf thing, this was a hack to avoid having to implement
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ok, I looked into how we could design this, and I think the best way to do this is to require queries that get fed to also get cached to disk. The only other alternative I came up with is to store all fed queries in the side effect table of their caller (somewhat hard, because we need to store all possible key/value combinations) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I no longer think this. That would require thinking about HIR in incremental, at minimum due to the |
||
} | ||
|
||
query inhabited_predicate_adt(key: DefId) -> ty::inhabitedness::InhabitedPredicate<'tcx> { | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -39,7 +39,7 @@ use rustc_hir::{HirId, Node, TraitCandidate}; | |
use rustc_index::IndexVec; | ||
use rustc_macros::{HashStable, TyDecodable, TyEncodable}; | ||
use rustc_query_system::cache::WithDepNode; | ||
use rustc_query_system::dep_graph::DepNodeIndex; | ||
use rustc_query_system::dep_graph::{DepNodeIndex, TaskDepsRef}; | ||
use rustc_query_system::ich::StableHashingContext; | ||
use rustc_serialize::opaque::{FileEncodeResult, FileEncoder}; | ||
use rustc_session::config::CrateType; | ||
|
@@ -50,10 +50,8 @@ use rustc_span::def_id::{CRATE_DEF_ID, DefPathHash, StableCrateId}; | |
use rustc_span::symbol::{Ident, Symbol, kw, sym}; | ||
use rustc_span::{DUMMY_SP, Span}; | ||
use rustc_type_ir::TyKind::*; | ||
use rustc_type_ir::fold::TypeFoldable; | ||
use rustc_type_ir::lang_items::TraitSolverLangItem; | ||
pub use rustc_type_ir::lift::Lift; | ||
use rustc_type_ir::{CollectAndApply, Interner, TypeFlags, WithCachedTypeInfo, search_graph}; | ||
use rustc_type_ir::WithCachedTypeInfo; | ||
use rustc_type_ir::{CollectAndApply, Interner, TypeFlags}; | ||
use tracing::{debug, instrument}; | ||
|
||
use crate::arena::Arena; | ||
|
@@ -1818,34 +1816,71 @@ impl<'tcx> TyCtxtAt<'tcx> { | |
|
||
impl<'tcx> TyCtxt<'tcx> { | ||
/// `tcx`-dependent operations performed for every created definition. | ||
#[instrument(level = "trace", skip(self))] | ||
pub fn create_def( | ||
self, | ||
parent: LocalDefId, | ||
name: Symbol, | ||
def_kind: DefKind, | ||
) -> TyCtxtFeed<'tcx, LocalDefId> { | ||
let data = def_kind.def_path_data(name); | ||
// The following call has the side effect of modifying the tables inside `definitions`. | ||
// The following create_def calls have the side effect of modifying the tables inside `definitions`. | ||
// These very tables are relied on by the incr. comp. engine to decode DepNodes and to | ||
// decode the on-disk cache. | ||
// | ||
// Any LocalDefId which is used within queries, either as key or result, either: | ||
// - has been created before the construction of the TyCtxt; | ||
// - has been created before the construction of the TyCtxt, | ||
// - has been created when marking a query as green (recreating definitions it created in the actual run), | ||
// - has been created by this call to `create_def`. | ||
// As a consequence, this LocalDefId is always re-created before it is needed by the incr. | ||
// comp. engine itself. | ||
// | ||
// This call also writes to the value of `source_span` and `expn_that_defined` queries. | ||
// This is fine because: | ||
// - those queries are `eval_always` so we won't miss their result changing; | ||
// - this write will have happened before these queries are called. | ||
Comment on lines
1835
to
1839
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Stale comment (pre-existing). |
||
let def_id = self.untracked.definitions.write().create_def(parent, data); | ||
|
||
// This function modifies `self.definitions` using a side-effect. | ||
// We need to ensure that these side effects are re-run by the incr. comp. engine. | ||
// Depending on the forever-red node will tell the graph that the calling query | ||
// needs to be re-evaluated. | ||
self.dep_graph.read_index(DepNodeIndex::FOREVER_RED_NODE); | ||
let def_id = tls::with_context(|icx| { | ||
match icx.task_deps { | ||
// Always gets rerun anyway, so nothing to replay | ||
TaskDepsRef::EvalAlways => { | ||
let def_id = self.untracked.definitions.write().create_def(parent, data).0; | ||
trace!(?def_id, "eval always"); | ||
def_id | ||
} | ||
// Top-level queries like the resolver get rerun every time anyway | ||
TaskDepsRef::Ignore => { | ||
let def_id = self.untracked.definitions.write().create_def(parent, data).0; | ||
trace!(?def_id, "ignore"); | ||
def_id | ||
} | ||
TaskDepsRef::Forbid => bug!( | ||
"cannot create definition {parent:?}, {name:?}, {def_kind:?} without being able to register task dependencies" | ||
), | ||
TaskDepsRef::Allow(_) => { | ||
let (def_id, hash) = | ||
self.untracked.definitions.write().create_def(parent, data); | ||
trace!(?def_id, "record side effects"); | ||
|
||
icx.side_effects.as_ref().unwrap().lock().definitions.push(DefIdInfo { | ||
parent, | ||
data, | ||
hash, | ||
}); | ||
def_id | ||
} | ||
TaskDepsRef::Replay { prev_side_effects, created_def_ids } => { | ||
trace!(?created_def_ids, "replay side effects"); | ||
trace!("num_defs : {}", prev_side_effects.definitions.len()); | ||
let index = created_def_ids.fetch_add(1, std::sync::atomic::Ordering::Relaxed); | ||
let prev_info = &prev_side_effects.definitions[index]; | ||
let def_id = self.untracked.definitions.read().local_def_path_hash_to_def_id( | ||
prev_info.hash, | ||
&"should have already recreated def id in try_mark_green", | ||
); | ||
assert_eq!(prev_info.data, data); | ||
assert_eq!(prev_info.parent, parent); | ||
def_id | ||
} | ||
Comment on lines
+1869
to
+1881
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This logic is subtle, we should definitely document it somewhere. Here, or on |
||
} | ||
}); | ||
|
||
let feed = TyCtxtFeed { tcx: self, key: def_id }; | ||
feed.def_kind(def_kind); | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this block necessary? The previous calls
tcx.ensure().eval_static_initializer
.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just test code to replicate the bug I had initially, it should not land