-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DEPS: bump across etcd-io/raft#81 and disable conf change validation #106515
Conversation
It looks like your PR touches production code but doesn't add or edit any test code. Did you consider adding tests to your PR? 🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf. |
b4ad7fc
to
2039d03
Compare
b8f81c8
to
4ecacde
Compare
Picks up - etcd-io/raft#77 - etcd-io/raft#79 (essentially an off-by-default for 77) - etcd-io/raft#82 - etcd-io/raft#81 (needed for cockroachdb#105797) Epic: none Release note: None
Relies on etcd-io/raft#81. We don't want it to, since that causes issues due to false positives. We are taking responsibility for carrying out only valid conf changes, as we always have. See also etcd-io/raft#80. Fixes cockroachdb#105797. Epic: CRDB-25287 Release note (bug fix): under rare circumstances, a replication change could get stuck when proposed near lease/leadership changes (and likely under overload), and the replica circuit breakers could trip. This problem has been addressed. Note to editors: this time it's really addressed (fingers crossed); a previous attempt with an identical release note had to be reverted.
4ecacde
to
4585dde
Compare
bors r=erikgrinaker |
Build succeeded: |
Encountered an error creating backports. Some common things that can go wrong:
You might need to create your backport manually using the backport tool. error creating merge commit from 404188d to blathers/backport-release-22.2-106515: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict [] you may need to manually resolve merge conflicts with the backport tool. Backport to branch 22.2.x failed. See errors above. error creating merge commit from 404188d to blathers/backport-release-23.1-106515: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict [] you may need to manually resolve merge conflicts with the backport tool. Backport to branch 23.1.x failed. See errors above. 🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf. |
@tbg @pav-kv now that we are using |
I had a similar concern when looking at this. I wonder if it would be sufficient to disable only the 2 checks here (which describe the state-machine-level rules for config transitions, which will be also checked by our state machine), but leave the one that @nvanbenschoten linked which is important for raft's config change safety. Seemingly, re-enabling this one check won't lead to the same perpetual reproposal behaviour for the config change, because the "leader has unapplied config changes" ( |
That seems like a reasonable change to me. Do you want to take that @pav-kv? |
124804: raft: re-enable config change safety r=nvanbenschoten a=pav-kv Config changes in this `raft` implementation require a safety constraint: the leader must not append a config change if it hasn't applied all config changes in its log. The `DisableConfChangeValidation` flag disables this check under the assumption that the state machine layer provides the equivalent guarantee. However, it is hard to argue that this is true in split leaseholder/leader scenarios. This commit re-enables this check, to bring the safety back. The other two state-machine-level checks concerned with entering and leaving joint configs can still be disabled. Related to #105797, etcd-io/raft#81, #106515 (#106515 (comment)) Epic: none Release note: none 124806: release: push PRs to main repo r=celiala a=rail Previously, we pushed PRs using the forked repos. For auto-merge to work, we need to push these PRs back to the main repo, so `GITHUB_TOKEN` has enough privileges to merge the PRs. This PR adds an ability to push the PR branches to the origin repo. Note: the push user has to be given `write` permissions to the repos. Epic: none Release note: None Co-authored-by: Pavel Kalinnikov <pavel@cockroachlabs.com> Co-authored-by: Rail Aliiev <rail@iqchoice.com>
We don't want raft to validate conf changes, since that causes issues due to false positives (the check is above raft, but needs to be below raft to always work correctly). We are
taking responsibility for carrying out only valid conf changes, as we always
have.
See also etcd-io/raft#80.
Fixes #105797.
Epic: CRDB-25287
Release note (bug fix): under rare circumstances, a replication change could get
stuck when proposed near lease/leadership changes (and likely under overload),
and the replica circuit breakers could trip. This problem has been addressed.
Note to editors: this time it's really addressed (fingers crossed); a previous
attempt with an identical release note had to be reverted.