-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Redwood chunked file growth and low priority IO starvation prevention #5936
Redwood chunked file growth and low priority IO starvation prevention #5936
Conversation
… reduce truncate() calls for expansion. PriorityMultiLock has limit on consecutive same-priority lock release. Increased Redwood max priority level to 3 for more separation at higher BTree levels.
…ss the simulated process has been set to have an unreliable disk.
…esponse to writes, which wait on only the necessary truncate operations. Increased buggified chunk size because truncate can be very slow in simulation.
…o-priority # Conflicts: # flow/genericactors.actor.h
…o-priority # Conflicts: # fdbserver/VersionedBTree.actor.cpp
…ait until at least the target timeout interval past the point when simulation is sped up.
…o-priority # Conflicts: # fdbclient/ServerKnobs.cpp # fdbclient/ServerKnobs.h # fdbserver/VersionedBTree.actor.cpp
… and do in place value updates if the new value is the same size as the old value.
AWS CodeBuild CI Report for macOS Catalina 10.15
|
AWS CodeBuild CI Report for macOS Catalina 10.15
|
AWS CodeBuild CI Report for Linux CentOS 7
|
AWS CodeBuild CI Report for Linux CentOS 7
|
…tchToLinearMerge() since it is only used in one place.
AWS CodeBuild CI Report for Linux CentOS 7
|
1 similar comment
AWS CodeBuild CI Report for Linux CentOS 7
|
AWS CodeBuild CI Report for Linux CentOS 7
|
5994501
to
80707a0
Compare
AWS CodeBuild CI Report for Linux CentOS 7
|
AWS CodeBuild CI Report for macOS Catalina 10.15
|
fdbserver/VersionedBTree.actor.cpp
Outdated
self->filePageCountPending = newPageCount; | ||
|
||
// Wait for any previous extensions to complete | ||
wait(self->fileExtension); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just to confirm my understanding, the reason this works along with line 2607 updating self->FileExtension to this future is because extendToCoverwill run up to the wait, including passing the current self->fileExtension to the wait, before updating self->fileExtension on line 2607?
This seems like a weird interaction with futures that might at least warrant a comment or something?
…n future as a parameter. Fixed initialization warning.
AWS CodeBuild CI Report for macOS Catalina 10.15
|
AWS CodeBuild CI Report for Linux CentOS 7
|
Redwood now grows files in chunks of 20k pages by default, controlled by a knob.
To prevent high read rates from starving lower priority write ops, PriorityMultiLock now
Simulation bug fix:
ioTimeoutError()
andioDegradedOrTimeoutError
won't trigger until afterspeedUpSimulation
is set since before then IO can be extremely slow in sim time.This PR also includes #5925.
The general guidelines can be found here.
Please check each of the following things and check all boxes before accepting a PR.
For Release-Branches
If this PR is made against a release-branch, please also check the following:
release-branch
ormaster
if this is the youngest branch)