kv: make RaftCommand.ClosedTimestamp nullable #60992
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This change is |
andreimatei
approved these changes
Feb 23, 2021
There was a problem hiding this comment.
thanks for the catch
Reviewable status:
complete! 1 of 0 LGTMs obtained (waiting on @nvanbenschoten)
pkg/kv/kvserver/kvserverpb/proposer_kv.proto, line 260 at r1 (raw file):
// This field is set through ClosedTimestampFooter hackery. Unlike in the // ClosedTimestampFooter, the field is nullable here so that it does not get // encoded when empty. This prevents the field from being encoded twice in the
want to put a similar comment on max_lease_index?
Fixes cockroachdb#60852. Fixes cockroachdb#60833. Fixes cockroachdb#58298. Fixes cockroachdb#59428. Fixes cockroachdb#60756. Fixes cockroachdb#60848. Fixes cockroachdb#60849. In cockroachdb#60852 and related issues, we saw that the introduction of a non-nullable `RaftCommand.ClosedTimestamp`, coupled with the `ClosedTimestampFooter` encoding strategy we use, led to encoded `RaftCommand` protos with their ClosedTimestamp field set twice. This is ok from a correctness perspective, at least as protobuf is concerned, but it led to a subtle interaction where the process of passing through sideloading (`maybeInlineSideloadedRaftCommand(maybeSideloadEntriesImpl(e))`) would reduce the size of an encoded RaftCommand by 3 bytes (the encoded size of an empty `hlc.Timestamp`). This was resulting in an `uncommittedSize` leak in Raft, which was eventually stalling on its `MaxUncommittedEntriesSize` limit. This commit fixes this issue by making `RaftCommand.ClosedTimestamp` nullable. With the field marked as nullable, it will no longer be encoded as an empty timestamp when unset, ensuring that when the encoded `ClosedTimestampFooter` is appended, it contains the only instance of the `ClosedTimestamp` field.
nvanbenschoten
force-pushed
the
nvanbenschoten/closedTSFieldNullable
branch
from
1740033
to
e833703
Compare
Now that `ClosedTimestampFooter` exists, be more specific and symmetric.
nvanbenschoten
force-pushed
the
nvanbenschoten/closedTSFieldNullable
branch
from
e833703
to
e46511a
Compare
nvanbenschoten
commented
Feb 23, 2021
There was a problem hiding this comment.
TFTR.
Reviewable status:
pkg/kv/kvserver/kvserverpb/proposer_kv.proto, line 260 at r1 (raw file):
Previously, andreimatei (Andrei Matei) wrote…
want to put a similar comment on
max_lease_index?
Done.
|
bors r+ |
|
This PR was included in a batch that was canceled, it will be automatically retried |
|
Build succeeded: |
This was referenced Feb 24, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #60852.
Fixes #60833.
Fixes #58298.
Fixes #59428.
Fixes #60756.
Fixes #60848.
Fixes #60849.
In #60852 and related issues, we saw that the introduction of a non-nullable
RaftCommand.ClosedTimestamp, coupled with theClosedTimestampFooterencoding strategy we use, led to encodedRaftCommandprotos with their ClosedTimestamp field set twice. This is ok from a correctness perspective, at least as protobuf is concerned, but it led to a subtle interaction where the process of passing through sideloading (maybeInlineSideloadedRaftCommand(maybeSideloadEntriesImpl(e))) would reduce the size of an encoded RaftCommand by 3 bytes (the encoded size of an emptyhlc.Timestamp). This was resulting in anuncommittedSizeleak in Raft, which was eventually stalling on itsMaxUncommittedEntriesSizelimit.This commit fixes this issue by making
RaftCommand.ClosedTimestampnullable. With the field marked as nullable, it will no longer be encoded as an empty timestamp when unset, ensuring that when the encodedClosedTimestampFooteris appended, it contains the only instance of theClosedTimestampfield.cc. @cockroachdb/bulk-io