-
Notifications
You must be signed in to change notification settings - Fork 4.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enable conservative out-of-bound snapshot cleaning #8811
Enable conservative out-of-bound snapshot cleaning #8811
Conversation
@@ -2965,10 +2965,9 @@ pub mod tests { | |||
} | |||
|
|||
#[test] | |||
fn test_accounts_purge_chained_purge_after_snapshot_restore_complex() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I understood this mystical test finally...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Moreover, https://github.com/solana-labs/solana/pull/8811/files#diff-2099c5256db4eb5975c8834af38f6456R1497 made this previous form of this test fail.
d372bdd
to
4cc6cee
Compare
Codecov Report
@@ Coverage Diff @@
## master #8811 +/- ##
======================================
Coverage 80.0% 80.0%
======================================
Files 265 265
Lines 57477 57481 +4
======================================
+ Hits 45998 46011 +13
+ Misses 11479 11470 -9 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
I ran v1.0 validator with this PR patched overnight, and it seems that it still produces valid snapshots. Overall, size reduction is 10x and restore-time reduction is 20x. That's amazing.
|
Also, snapshot creation-time reduction is 20x (10min -> 30s): grafana-testnet-monitor-edge-ryoqun-without-out-of-bound-snaphost-cleaning.pdf |
That is amazing!! |
* Enable conservative out-of-bound snapshot cleaning * Add tests (cherry picked from commit 4bbf09f)
I also tested against the SLP cluster. And there was no problem, too. |
Problem
There is no codepath for cleaning unused storages in snapshot when starting a validator
Summary of Changes
Our past efforts has paid off; Now just need small plumbing and test clarification.
And, I'm now very conservative with good reason of newly-introduced bugs with the adventure of #8168. So, this PR doesn't contain aggressive strategy, originally had in #8337.
Let it rip little by little. :)
Not codewise, but this PR requires #8724
Split from #8337
Towards #8168