-
Notifications
You must be signed in to change notification settings - Fork 665
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: ensure unused KV and Cache blobs cleaned up #4466
Conversation
🦋 Changeset detectedLatest commit: 61c4b0b The changes in this PR will be included in the next version bump. This PR includes changesets to release 3 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
A wrangler prerelease is available for testing. You can install this latest build in your project with: npm install --save-dev https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/7085442368/npm-package-wrangler-4466 You can reference the automatically updated head of this PR with: npm install --save-dev https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/prs/7085442368/npm-package-wrangler-4466 Or you can use npx https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/7085442368/npm-package-wrangler-4466 dev path/to/script.js Additional artifacts:npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/7085442368/npm-package-miniflare-4466 npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/7085442368/npm-package-cloudflare-pages-shared-4466 npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/7085442368/npm-package-create-cloudflare-4466 Note that these links will no longer work once the GitHub Actions artifact expires.
| Please ensure constraints are pinned, and |
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## main #4466 +/- ##
==========================================
+ Coverage 75.46% 75.48% +0.01%
==========================================
Files 225 240 +15
Lines 12478 12854 +376
Branches 3239 3312 +73
==========================================
+ Hits 9417 9703 +286
- Misses 3061 3151 +90 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
cloudflare/miniflare#656 introduced a bug in the SQL statement used for getting the old blob ID of entries to delete when overriding keys. This meant if a key was overridden, the old blob pointed to by that key was not deleted. This lead to an accumulation of garbage when `kvPersist` and `cachePersist` were enabled.
26f50d3
to
61c4b0b
Compare
What this PR solves / how to test:
cloudflare/miniflare#656 introduced a bug in the SQL statement used for getting the old blob ID of entries to delete when overriding keys. This meant if a key was overridden, the old blob pointed to by that key was not deleted. This lead to an accumulation of garbage when
kvPersist
andcachePersist
were enabled. This shouldn't have caused any problems, but could lead to persistence directories growing unnecessarily as they filled up with garbage blobs. This change ensures garbage blobs are deleted, and adds some regression tests to prevent this happening again.Note existing garbage will not be cleaned up. If you'd like to do this, download this Node script (https://gist.github.com/mrbbot/68787e19dcde511bd99aa94997b39076). If you're using the default Wrangler persistence directory, run
node gc.mjs kv .wrangler/state/v3/kv <namespace_id_1> <namespace_id_2> ...
andnode gc.mjs cache .wrangler/state/v3/cache default named:<cache_name_1> named:<cache_name_2> ...
with each of your KV namespace IDs (not binding names) and named caches.Author has addressed the following:
Note for PR author:
We want to celebrate and highlight awesome PR review! If you think this PR received a particularly high-caliber review, please assign it the label
highlight pr review
so future reviewers can take inspiration and learn from it.