Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set fork solution as preference when resolving #4662

Merged
merged 1 commit into from
Jul 1, 2024
Merged

Conversation

charliermarsh
Copy link
Member

Summary

This should both make it faster to solve forks (since we have a guess for a valid resolution, and will bias towards packages we've already fetched) and improve consistency between forks.

Closes #4617.

@@ -99,7 +99,7 @@ impl Preference {

/// A set of pinned packages that should be preserved during resolution, if possible.
#[derive(Debug, Clone, Default)]
pub struct Preferences(Arc<FxHashMap<PackageName, Pin>>);
pub struct Preferences(FxHashMap<PackageName, Pin>);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does anyone know why this is in an Arc?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was added when moving the solver into its own thread @ibraheemdev


// Walk over the selected versions, and mark them as preferences.
for state in &mut forked_states {
for (package, versions) in &resolution.packages {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is quadratic but presumedly there are a small number of forks...?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about updating the ResolverState preferences instead? This should update all forks implicitly.

@@ -1733,8 +1707,7 @@ fn fork_marker_selection() -> Result<()> {
version = "0.1.0"
source = { editable = "." }
dependencies = [
{ name = "package-a", version = "0.1.0", source = { registry = "https://astral-sh.github.io/packse/0.3.29/simple-html/" } },
{ name = "package-a", version = "0.2.0", source = { registry = "https://astral-sh.github.io/packse/0.3.29/simple-html/" } },
{ name = "package-a" },
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment on this test says:

/// So this acts as a regression test to ensure that only one version of `a` is selected.

But I guess it wasn't actually doing that!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I think we need a way to assert the result in the packse scenario itself. Otherwise it's pretty easy to miss if the result of the test isn't what you expected it to be.

@@ -99,7 +99,7 @@ impl Preference {

/// A set of pinned packages that should be preserved during resolution, if possible.
#[derive(Debug, Clone, Default)]
pub struct Preferences(Arc<FxHashMap<PackageName, Pin>>);
pub struct Preferences(FxHashMap<PackageName, Pin>);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was added when moving the solver into its own thread @ibraheemdev


// Walk over the selected versions, and mark them as preferences.
for state in &mut forked_states {
for (package, versions) in &resolution.packages {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about updating the ResolverState preferences instead? This should update all forks implicitly.

@charliermarsh
Copy link
Member Author

What about updating the ResolverState preferences instead? This should update all forks implicitly.

That would require that the resolver state is taken mutably, but it's not here.

@charliermarsh
Copy link
Member Author

I could do it by wrapping Preferences in a Mutex or RwLock but that may have bigger perf implications - wdyt?

@charliermarsh charliermarsh merged commit 2d57309 into main Jul 1, 2024
47 checks passed
@charliermarsh charliermarsh deleted the charlie/pref branch July 1, 2024 12:25
@konstin
Copy link
Member

konstin commented Jul 1, 2024

We could try a mutable Preferences outside the 'FORK scope that we pass around, this should get us the same effect without any performance implications.

@charliermarsh
Copy link
Member Author

That's true, I can try that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Potential performance improvement preview Experimental behavior
Projects
None yet
Development

Successfully merging this pull request may close these issues.

universal-lock: use resolution of a fork as input preferences to its child forks
3 participants