Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

domain, store/tikv: enable etcd client auto sync #9575

Merged
merged 2 commits into from
Mar 7, 2019
Merged

domain, store/tikv: enable etcd client auto sync #9575

merged 2 commits into from
Mar 7, 2019

Conversation

disksing
Copy link
Contributor

@disksing disksing commented Mar 6, 2019

Signed-off-by: disksing i@disksing.com

What problem does this PR solve?

When all PD nodes are replaced by new PD nodes, etcd clients (used by DDL worker and safepoint syncer) in tidb will lost connection with PD cluster.

What is changed and how it works?

Enable etcd's AutoSync feature so it will automatically update member list.

Check List

Tests

  • Manual test (add detailed scripts or steps below)
  1. start a cluster with 3 PDs, 1 TiKV, 1TiDB
  2. Join 3 more PDs then member delete original 3 PDs.
  3. Wait 100s and check if there are any 'safePoint out of sync' errors (TRUE for master, FALSE for auto-sync)
  4. Run some DDL and check if it succeds (FALSE for master, TRUE for auto-sync)

Note: No unit test case included, it will be more suitable to add a integration test in schrodinger later. This bug is kind urgent and the fix is straightforward so I guess we can merge it ahead.

Related changes

  • Need to cherry-pick to the release branch
  • Need to be included in the release note

Signed-off-by: disksing <i@disksing.com>
@disksing disksing added the type/bugfix This PR fixes a bug. label Mar 6, 2019
@codecov-io
Copy link

codecov-io commented Mar 6, 2019

Codecov Report

Merging #9575 into master will decrease coverage by <.01%.
The diff coverage is 0%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #9575      +/-   ##
==========================================
- Coverage    67.3%   67.29%   -0.01%     
==========================================
  Files         376      376              
  Lines       78961    78963       +2     
==========================================
- Hits        53144    53139       -5     
- Misses      21058    21061       +3     
- Partials     4759     4763       +4
Impacted Files Coverage Δ
store/tikv/kv.go 53.52% <0%> (-0.26%) ⬇️
domain/domain.go 40.24% <0%> (-0.08%) ⬇️
ddl/delete_range.go 75.13% <0%> (-4.24%) ⬇️
store/tikv/gcworker/gc_worker.go 41.57% <0%> (+0.42%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f8d9020...4f02e67. Read the comment docs.

Copy link
Contributor

@zimulala zimulala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zimulala zimulala added the status/LGT1 Indicates that a PR has LGTM 1. label Mar 6, 2019
Copy link
Member

@nolouch nolouch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zz-jason zz-jason merged commit 108c3d2 into pingcap:master Mar 7, 2019
@disksing disksing deleted the auto-sync branch March 7, 2019 09:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status/LGT1 Indicates that a PR has LGTM 1. type/bugfix This PR fixes a bug.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants