Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

shink all slots in parallel on startup #17308

Merged
merged 1 commit into from
May 19, 2021

Conversation

jeffwashington
Copy link
Contributor

@jeffwashington jeffwashington commented May 18, 2021

Problem

validator startup time is painful - especially as # accounts grows. There will soon be an issue filed for this.

Summary of Changes

From verify_snapshot_bank, shrink slots in parallel.
Fixes #

@jeffwashington
Copy link
Contributor Author

jeffwashington commented May 18, 2021

shrink_all_slots_us

verify_snapshot_bank clean_us=57710861i shrink_all_slots_us=1097525i verify_bank_hash_us=19499188i verify_hash_us=2205i (master)
verify_snapshot_bank clean_us=57490783i shrink_all_slots_us= 622640i verify_bank_hash_us=19254628i verify_hash_us=2209i (this pr)

@jeffwashington jeffwashington changed the title shink all in parallel on startup shink all slots in parallel on startup May 18, 2021
@jeffwashington jeffwashington requested a review from carllin May 18, 2021 18:44
@codecov
Copy link

codecov bot commented May 18, 2021

Codecov Report

Merging #17308 (4cbd83f) into master (a3c0833) will decrease coverage by 0.0%.
The diff coverage is 81.8%.

@@            Coverage Diff            @@
##           master   #17308     +/-   ##
=========================================
- Coverage    82.7%    82.7%   -0.1%     
=========================================
  Files         421      421             
  Lines      118064   118076     +12     
=========================================
- Hits        97687    97667     -20     
- Misses      20377    20409     +32     

runtime/src/bank.rs Outdated Show resolved Hide resolved
} else {
self.do_shrink_slot_forced_v1(slot);
pub fn shrink_all_slots(&self, startup: bool) {
if startup && self.caching_enabled {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any particular reason we can't do the parallel shrink when caching is not enabled?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't want to test that code path or be worried about it ;-)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are actively maintaining and improving the caching code path. I have more confidence we will get that codepath right and keep it right.

@jeffwashington jeffwashington requested a review from carllin May 18, 2021 20:44
@jeffwashington jeffwashington marked this pull request as ready for review May 18, 2021 20:44
@jeffwashington jeffwashington merged commit c20b27b into solana-labs:master May 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants