partition-allocator: default shard 0 reserved partitions to 0 #22841
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We have the topic_partitions_reserve_shard0 property which makes the partition allocator have as if there are that many partitions allocated to shard0, beyond the amount actually allocated. The idea is to slightly bias the partition related work on shard 0 in order to account for extra duties it has to do. This is by default set to 2. This change sets it to 0 instead. In general the default value of 2 does not do a good job of accounting for the bias: it will either be much too high or much too low in most cases. I.e., if every shard has 500 partitions, putting "only" 498 makes no difference, despite that shard 0 might be doing significantly more work in this case. On the other hand, if someone takes our "recommendation" of using 1 or 2 shards per partition for maximum tput, they might be surprised to find that shard 0 get no partitions at all, since it has these two reserved slots. In this case shard 0 will be seriously underutilized. Especially due to this second effect, which does come up in practice, it seems like 0 is the best default value for this property. The idea of choosing a fixed value to reduce the load on shard 0 seems appealing on the surface but does not seem to work out in practice. CORE-6852
travisdowns
changed the title
|
ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/52779#019142c8-e4a7-4bf4-b7cc-ad04cd0d660c |
StephanDollberg
approved these changes
Aug 12, 2024
mmaslankaprv
approved these changes
Aug 13, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We have the topic_partitions_reserve_shard0 property which makes the partition allocator have as if there are that many partitions allocated to shard0, beyond the amount actually allocated. The idea is to slightly bias the partition related work on shard 0 in order to account for extra duties it has to do.
This is by default set to 2. This change sets it to 0 instead. In general the default value of 2 does not do a good job of accounting for the bias: it will either be much too high or much too low in most cases. I.e., if every shard has 500 partitions, putting "only" 498 makes no difference, despite that shard 0 might be doing significantly more work in this case. On the other hand, if someone takes our "recommendation" of using 1 or 2 shards per partition for maximum tput, they might be surprised to find that shard 0 get no partitions at all, since it has these two reserved slots. In this case shard 0 will be seriously underutilized.
Especially due to this second effect, which does come up in practice, it seems like 0 is the best default value for this property. The idea of choosing a fixed value to reduce the load on shard 0 seems appealing on the surface but does not seem to work out in practice.
CORE-6852
Backports Required
Release Notes
Improvements
topic_partitions_reserve_shard0to zero. This means that we no longer weight shard 0 as if it has 2 more partitions than it actually has, leading to more even partition distribution in cases where the total number of partitions is close to the vCPU count.