Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tikv: remove the update leader backoff when known real leader (#17541) #17681

Merged
merged 2 commits into from
Jun 6, 2020

Conversation

sre-bot
Copy link
Contributor

@sre-bot sre-bot commented Jun 4, 2020

cherry-pick #17541 to release-4.0


Signed-off-by: Liqi Geng gengliqiii@gmail.com

What problem does this PR solve?

Issue Number: tikv/tikv#7941

Problem Summary:
When a down TiKV recovers, PD will transfer many leaders to this TiKV. I found it will cause that QPS recovers so slowly because of TiDB backoff strategy.
If a region's leader has transferred, the request to this region may get a NotLeader error with a correct leader in most cases. Then TiDB will backoff at least 1ms(then 2,4,8,10ms) which is very wasteful.
In tikv/tikv#7941, it's showed that the QPS recover time is reduced to 2-4min if we remove the update leader backoff, while the original time is 10min.

What is changed and how it works?

What's Changed:
Remove the update leader backoff.

How it Works:

Related changes

  • Need to cherry-pick to the release branch

Check List

Tests

  • No code

Side effects

No.

Release note

  • Mitigate the impact of failure recovery on QPS.

Signed-off-by: sre-bot <sre-bot@pingcap.com>
@sre-bot
Copy link
Contributor Author

sre-bot commented Jun 4, 2020

/run-all-tests

Copy link
Contributor

@lysu lysu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lysu lysu added the status/LGT1 Indicates that a PR has LGTM 1. label Jun 5, 2020
@bb7133 bb7133 modified the milestones: v4.0.1, v4.0.2 Jun 6, 2020
Copy link
Member

@zz-jason zz-jason left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zz-jason zz-jason added status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. status/PTAL labels Jun 6, 2020
@sre-bot
Copy link
Contributor Author

sre-bot commented Jun 6, 2020

Your auto merge job has been accepted, waiting for:

  • 17694
  • 17680

@sre-bot
Copy link
Contributor Author

sre-bot commented Jun 6, 2020

/run-all-tests

@sre-bot sre-bot merged commit cf7467a into pingcap:release-4.0 Jun 6, 2020
@lysu lysu changed the title tikv: remove the update leader backoff (#17541) tikv: remove the update leader backoff when known real leader (#17541) Apr 16, 2021
@ti-chi-bot ti-chi-bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Apr 16, 2021
@zhouqiang-cl zhouqiang-cl modified the milestones: v4.0.2, v4.0.13 May 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/tikv size/M Denotes a PR that changes 30-99 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. type/enhancement The issue or PR belongs to an enhancement. type/4.0-cherry-pick
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants