Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: invalidate cache for out-of-scope entries #104

Merged
merged 1 commit into from
Apr 17, 2024
Merged

Conversation

klkvr
Copy link
Member

@klkvr klkvr commented Apr 16, 2024

missed this in #90

ref https://t.me/foundry_support/52331

after #90 we stopped removing out of scope sources from cache, however, we never validated if they are dirty, this results in invalid cached artifacts in certain cases.

with this PR we iterate over all cache entries out of scope of current compiler run, then build graph with them to find if they import any of files marked as dirty.

@klkvr klkvr requested a review from mattsse April 16, 2024 21:05
Copy link
Member

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice,
triggering patch once merged

Comment on lines +729 to +730
// Build a temporary graph to repeat the process above. We need this because `self.edges`
// only contains graph data for in-scope sources.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, yeah, that makes sense

@mattsse mattsse merged commit a086b9d into main Apr 17, 2024
14 checks passed
@klkvr klkvr deleted the klkvr/cache-fix branch April 17, 2024 11:54
@klkvr klkvr mentioned this pull request Apr 17, 2024
mattsse pushed a commit that referenced this pull request Apr 17, 2024
Follow-up to #104. Right now we detect and remove dirty cache entries in
`ArtifactsCacheInner::filter` which is redundant as we can just do a
single iteration over all cached files.

Now file filtering is done in 2 steps:
1. `ArtifactsCacheInner::find_and_remove_dirty` detects all dirty
entries (even those that are not in scope of current compiler run) and
removes them from cache.
2. `ArtifactsCacheInner::filter` schedules for compilation all sources
for which we are missing artifacts (either because of removed entry or
because we never compiled given source with current version)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants