Update epoch slots to include all missing slots #8276
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pgarg66
commented
Feb 14, 2020
Codecov Report
@@ Coverage Diff @@
## master #8276 +/- ##
========================================
+ Coverage 80.6% 80.6% +<.1%
========================================
Files 253 253
Lines 55247 55386 +139
========================================
+ Hits 44550 44683 +133
- Misses 10697 10703 +6 |
carllin
reviewed
Feb 14, 2020
core/src/cluster_info.rs
Outdated
| .expect("expected to find at least one slot"); | ||
| let last_slot = incomplete_slots | ||
| .iter() | ||
| .last() |
There was a problem hiding this comment.
Because this iterator implements DoubleEndedIterator , next_back() is a more efficient way to get the last item without iterating over the entire set
There was a problem hiding this comment.
LMK when you are done with the review. I'll update the PR with the required changes.
mergify
bot
dismissed
aeyakovenko’s stale review
Pull request has been modified.
|
@carllin , the latest patch should start limiting the cache size, instead of limiting it based on roots. @aeyakovenko , there are two separate changes we can make down the line.
|
|
@pgarg66 we just need to do 1. And replace the oldest stash. This would also reduce the churn in gossip |
mergify bot
pushed a commit
that referenced
this pull request
Feb 20, 2020
* Update epoch slots to include all missing slots * new test for compress/decompress * address review comments * limit cache based on size, instead of comparing roots (cherry picked from commit 0d5c123) # Conflicts: # Cargo.lock
pgarg66
added a commit
to pgarg66/solana
that referenced
this pull request
Feb 20, 2020
* Update epoch slots to include all missing slots * new test for compress/decompress * address review comments * limit cache based on size, instead of comparing roots
pgarg66
added a commit
that referenced
this pull request
Feb 20, 2020
* Update epoch slots to include all missing slots * new test for compress/decompress * address review comments * limit cache based on size, instead of comparing roots
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
Some shreds could be missing across the cluster (e.g. due to turbine failure). Such shreds will be repaired by all/most nodes. Need a mechanism to detect that most of the cluster is missing shreds/slots, so that the such shreds could be pushed thru turbine again.
Summary of Changes
This change augments epoch_slots information that's pushed thru gossip. It'll include information about all incomplete slots in the current epoch. Since the list could be large, it's getting compressed using gzip.