-
Notifications
You must be signed in to change notification settings - Fork 155
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Assign memo ingredients per salsa-struct-ingredient #614
Changes from 5 commits
fac6053
2cae1ef
d34ed14
738d5f9
38ad655
7aa1518
53c7eba
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -99,6 +99,9 @@ macro_rules! setup_tracked_fn { | |
$zalsa::IngredientCache::new(); | ||
|
||
impl $zalsa::SalsaStructInDb for $InternedData<'_> { | ||
fn lookup_ingredient_index(_aux: &dyn $zalsa::JarAux) -> core::option::Option<$zalsa::IngredientIndex> { | ||
None | ||
} | ||
} | ||
|
||
impl $zalsa::interned::Configuration for $Configuration { | ||
|
@@ -199,7 +202,19 @@ macro_rules! setup_tracked_fn { | |
aux: &dyn $zalsa::JarAux, | ||
first_index: $zalsa::IngredientIndex, | ||
) -> Vec<Box<dyn $zalsa::Ingredient>> { | ||
let struct_index = $zalsa::macro_if! { | ||
if $needs_interner { | ||
first_index.successor(0) | ||
} else { | ||
<$InternedData as $zalsa::SalsaStructInDb>::lookup_ingredient_index(aux) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. OK, I see now the role of |
||
.expect( | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not 1000% percent sure that the struct ingredient will always have been created first. If this code doesn't run until the tracked functon is first called, I suppose that's true, but if it could run at some other time maybe not? Still, this seems like a good assertion for now to simplify our lives. If it's not true we can probably add some kind of dependency mechanism to force it to be true. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I thought - and experimented with some cases - that the structs are created first to be passed to a tracked function but my understanding on salsa is very limited and I might be wrong. I'll investigate this more |
||
"Salsa struct is passed as an argument of a tracked function, but its ingredient hasn't been added!" | ||
) | ||
} | ||
}; | ||
|
||
let fn_ingredient = <$zalsa::function::IngredientImpl<$Configuration>>::new( | ||
struct_index, | ||
first_index, | ||
aux, | ||
); | ||
|
@@ -219,6 +234,10 @@ macro_rules! setup_tracked_fn { | |
} | ||
} | ||
} | ||
|
||
fn salsa_struct_type_id(&self) -> Option<core::any::TypeId> { | ||
None | ||
} | ||
} | ||
|
||
#[allow(non_local_definitions)] | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1,5 @@ | ||
pub trait SalsaStructInDb {} | ||
use crate::{plumbing::JarAux, IngredientIndex}; | ||
|
||
pub trait SalsaStructInDb { | ||
fn lookup_ingredient_index(aux: &dyn JarAux) -> Option<IngredientIndex>; | ||
} |
Original file line number | Diff line number | Diff line change | ||||||||
---|---|---|---|---|---|---|---|---|---|---|
@@ -1,5 +1,5 @@ | ||||||||||
use append_only_vec::AppendOnlyVec; | ||||||||||
use parking_lot::Mutex; | ||||||||||
use parking_lot::{Mutex, RwLock}; | ||||||||||
use rustc_hash::FxHashMap; | ||||||||||
use std::any::{Any, TypeId}; | ||||||||||
use std::marker::PhantomData; | ||||||||||
|
@@ -119,8 +119,10 @@ pub struct Zalsa { | |||||||||
|
||||||||||
nonce: Nonce<StorageNonce>, | ||||||||||
|
||||||||||
/// Number of memo ingredient indices created by calls to [`next_memo_ingredient_index`](`Self::next_memo_ingredient_index`) | ||||||||||
memo_ingredients: Mutex<Vec<IngredientIndex>>, | ||||||||||
/// Map from the [`IngredientIndex::as_usize`][] of a salsa struct to a list of | ||||||||||
/// [ingredient-indices](`IngredientIndex`) for tracked functions that have this salsa struct | ||||||||||
/// as input. | ||||||||||
memo_ingredient_indices: RwLock<Vec<Vec<IngredientIndex>>>, | ||||||||||
|
||||||||||
/// Map from the type-id of an `impl Jar` to the index of its first ingredient. | ||||||||||
/// This is using a `Mutex<FxHashMap>` (versus, say, a `FxDashMap`) | ||||||||||
|
@@ -152,7 +154,7 @@ impl Zalsa { | |||||||||
ingredients_vec: AppendOnlyVec::new(), | ||||||||||
ingredients_requiring_reset: AppendOnlyVec::new(), | ||||||||||
runtime: Runtime::default(), | ||||||||||
memo_ingredients: Default::default(), | ||||||||||
memo_ingredient_indices: Default::default(), | ||||||||||
} | ||||||||||
} | ||||||||||
|
||||||||||
|
@@ -186,21 +188,22 @@ impl Zalsa { | |||||||||
{ | ||||||||||
let jar_type_id = jar.type_id(); | ||||||||||
let mut jar_map = self.jar_map.lock(); | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
*jar_map | ||||||||||
.entry(jar_type_id) | ||||||||||
.or_insert_with(|| { | ||||||||||
let index = IngredientIndex::from(self.ingredients_vec.len()); | ||||||||||
let ingredients = jar.create_ingredients(self, index); | ||||||||||
let mut should_create = false; | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't quite follow this -- why move this logic out from the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I guess it is so that ingredient construction can call There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It'd be nice to have a comment here explaining that interaction There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't remember for sure - as it was weeks before 😅 -, but I think that it was because it had some issues due to creation order like using ingredients index before they are created. |
||||||||||
let index = *jar_map.entry(jar_type_id).or_insert_with(|| { | ||||||||||
nikomatsakis marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||
should_create = true; | ||||||||||
IngredientIndex::from(self.ingredients_vec.len()) | ||||||||||
}); | ||||||||||
if should_create { | ||||||||||
let aux = JarAuxImpl(self, &jar_map); | ||||||||||
let ingredients = jar.create_ingredients(&aux, index); | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There's a comment on So moving it out to avoid the deadlock might not be entirely safe There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. For example. What could happen now is that two calls end up with the same index because the new ingredient isn't pushed on the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Okay, I'll fix this |
||||||||||
for ingredient in ingredients { | ||||||||||
let expected_index = ingredient.ingredient_index(); | ||||||||||
|
||||||||||
if ingredient.requires_reset_for_new_revision() { | ||||||||||
self.ingredients_requiring_reset.push(expected_index); | ||||||||||
} | ||||||||||
|
||||||||||
let actual_index = self | ||||||||||
.ingredients_vec | ||||||||||
.push(ingredient); | ||||||||||
let actual_index = self.ingredients_vec.push(ingredient); | ||||||||||
assert_eq!( | ||||||||||
expected_index.as_usize(), | ||||||||||
actual_index, | ||||||||||
|
@@ -209,10 +212,10 @@ impl Zalsa { | |||||||||
expected_index, | ||||||||||
actual_index, | ||||||||||
); | ||||||||||
|
||||||||||
} | ||||||||||
index | ||||||||||
}) | ||||||||||
} | ||||||||||
|
||||||||||
index | ||||||||||
} | ||||||||||
} | ||||||||||
|
||||||||||
|
@@ -290,15 +293,34 @@ impl Zalsa { | |||||||||
|
||||||||||
pub(crate) fn ingredient_index_for_memo( | ||||||||||
&self, | ||||||||||
struct_ingredient_index: IngredientIndex, | ||||||||||
memo_ingredient_index: MemoIngredientIndex, | ||||||||||
) -> IngredientIndex { | ||||||||||
self.memo_ingredients.lock()[memo_ingredient_index.as_usize()] | ||||||||||
self.memo_ingredient_indices.read()[struct_ingredient_index.as_usize()] | ||||||||||
[memo_ingredient_index.as_usize()] | ||||||||||
} | ||||||||||
} | ||||||||||
|
||||||||||
impl JarAux for Zalsa { | ||||||||||
fn next_memo_ingredient_index(&self, ingredient_index: IngredientIndex) -> MemoIngredientIndex { | ||||||||||
let mut memo_ingredients = self.memo_ingredients.lock(); | ||||||||||
struct JarAuxImpl<'a>(&'a Zalsa, &'a FxHashMap<TypeId, IngredientIndex>); | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I changed salsa struct ingredient lookup as I said here but I had to change this to avoid deadlocks 😢 |
||||||||||
|
||||||||||
impl JarAux for JarAuxImpl<'_> { | ||||||||||
fn lookup_jar_by_type(&self, jar: &dyn Jar) -> Option<IngredientIndex> { | ||||||||||
self.1.get(&jar.type_id()).map(ToOwned::to_owned) | ||||||||||
} | ||||||||||
|
||||||||||
fn next_memo_ingredient_index( | ||||||||||
&self, | ||||||||||
struct_ingredient_index: IngredientIndex, | ||||||||||
ingredient_index: IngredientIndex, | ||||||||||
) -> MemoIngredientIndex { | ||||||||||
let mut memo_ingredients = self.0.memo_ingredient_indices.write(); | ||||||||||
let idx = struct_ingredient_index.as_usize(); | ||||||||||
let memo_ingredients = if let Some(memo_ingredients) = memo_ingredients.get_mut(idx) { | ||||||||||
memo_ingredients | ||||||||||
} else { | ||||||||||
memo_ingredients.resize_with(idx + 1, Vec::new); | ||||||||||
memo_ingredients.get_mut(idx).unwrap() | ||||||||||
}; | ||||||||||
let mi = MemoIngredientIndex(u32::try_from(memo_ingredients.len()).unwrap()); | ||||||||||
memo_ingredients.push(ingredient_index); | ||||||||||
mi | ||||||||||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
//! Test that a `tracked` fn on multiple salsa struct args | ||
//! compiles and executes successfully. | ||
#[salsa::input] | ||
struct MyInput { | ||
field: u32, | ||
} | ||
|
||
#[salsa::interned] | ||
struct MyInterned<'db> { | ||
field: u32, | ||
} | ||
|
||
#[salsa::tracked] | ||
fn tracked_fn<'db>(db: &'db dyn salsa::Database, input: MyInput, interned: MyInterned<'db>) -> u32 { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why this particular test? Tracked functions with multiple arguments today compile to a tracked function on a hidden interned value -- is that what you were intending to test? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes. Though this PR is not directly related to that, but as it changes some codes in such hidden interned value, I was intending to add a regression test for it. Other case are pre-existing |
||
input.field(db) + interned.field(db) | ||
} | ||
|
||
#[test] | ||
fn execute() { | ||
let db = salsa::DatabaseImpl::new(); | ||
let input = MyInput::new(&db, 22); | ||
let interned = MyInterned::new(&db, 33); | ||
assert_eq!(tracked_fn(&db, input, interned), 55); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I...guess this is fine. It's a bit surprising to use the index of something that hasn't been made yet. An alternative would be to flip the order so that we create the interned struct first and then pass its index to the function. There'd probably be some minor adjustments made elsewhere in the file as a result but shouldn't be too bad.