Avoid growing notifications map during recovery #385
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
As this could negatively affect recovery performance of long logs.
|
Nice one! In my test scenario - a quorum queue with 10 million 5kb messages, the recovery process on node startup went down from ~250s to ~170s. On top of that, the memory consumption during recovery went down as well as expected. Here's the memory usage during the node startup (with just that one queue), first with this PR, then with |
mkuratczyk
added a commit
to rabbitmq/rabbitmq-server
that referenced
this pull request
Jul 13, 2023
Improved long log recovery: faster and lower memory usage. more info: rabbitmq/ra#385
mergify bot
pushed a commit
to rabbitmq/rabbitmq-server
that referenced
this pull request
Jul 13, 2023
Improved long log recovery: faster and lower memory usage. more info: rabbitmq/ra#385 (cherry picked from commit 805f679)
mergify bot
pushed a commit
to rabbitmq/rabbitmq-server
that referenced
this pull request
Jul 13, 2023
Improved long log recovery: faster and lower memory usage. more info: rabbitmq/ra#385 (cherry picked from commit 805f679) (cherry picked from commit 2c21b65)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As this could negatively affect recovery performance of long logs.
Proposed Changes
Please describe the big picture of your changes here to communicate to the
RabbitMQ team why we should accept this pull request. If it fixes a bug or
resolves a feature request, be sure to link to that issue.
A pull request that doesn't explain why the change was made has a much
lower chance of being accepted.
If English isn't your first language, don't worry about it and try to
communicate the problem you are trying to solve to the best of your abilities.
As long as we can understand the intent, it's all good.
Types of Changes
What types of changes does your code introduce to this project?
Put an
xin the boxes that applyChecklist
Put an
xin the boxes that apply. You can also fill these out after creatingthe PR. If you're unsure about any of them, don't hesitate to ask on the
mailing list. We're here to help! This is simply a reminder of what we are
going to look for before merging your code.
CONTRIBUTING.mddocumentFurther Comments
If this is a relatively large or complex change, kick off the discussion by
explaining why you chose the solution you did and what alternatives you
considered, etc.