Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tikv: fix infinite retry when kv region continuing to return StaleCommand error (#16481) #16530

Merged
merged 3 commits into from
Apr 22, 2020

Conversation

sre-bot
Copy link
Contributor

@sre-bot sre-bot commented Apr 17, 2020

cherry-pick #16481 to release-4.0


What problem does this PR solve?

Issue Number: close #16524

Problem Summary:

tikv region will report staleCommand when it can not catch up the log and will let tidb do retry.

but stale error didn't use backoff util, so it will infinite loop when some kv region keep return stale.

it can be reproduced by TestOnRegionError

this will make SQLs block forever if it hit those regions.

What is changed and how it works?

What's Changed:

do a little backoff before retry stale command,

How it Works:

do backoff, let retry loop has chances be broken by maxSleepTime or kill command

Related changes

  • Need to cherry-pick to the release branch(3.x need modify parser)

Check List

Tests

  • Unit test

Side effects

  • n/a

Release note

fix infinite retry when kv continuing to return staleCommand error


This change is Reviewable

@sre-bot
Copy link
Contributor Author

sre-bot commented Apr 17, 2020

/run-all-tests

@sre-bot
Copy link
Contributor Author

sre-bot commented Apr 19, 2020

@coocood, @tiancaiamao, @jackysp, @crazycs520, PTAL.

@sre-bot
Copy link
Contributor Author

sre-bot commented Apr 21, 2020

@coocood, @tiancaiamao, @jackysp, @crazycs520, PTAL.

@tiancaiamao
Copy link
Contributor

LGTM

@tiancaiamao tiancaiamao added the status/LGT1 Indicates that a PR has LGTM 1. label Apr 21, 2020
@lysu
Copy link
Contributor

lysu commented Apr 22, 2020

/merge

@sre-bot
Copy link
Contributor Author

sre-bot commented Apr 22, 2020

Sorry @lysu, you don't have permission to trigger auto merge event on this branch.

@jackysp
Copy link
Member

jackysp commented Apr 22, 2020

/merge

@sre-bot sre-bot added the status/can-merge Indicates a PR has been approved by a committer. label Apr 22, 2020
@sre-bot
Copy link
Contributor Author

sre-bot commented Apr 22, 2020

/run-all-tests

@sre-bot sre-bot merged commit dcbe4e2 into pingcap:release-4.0 Apr 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/tikv status/can-merge Indicates a PR has been approved by a committer. status/LGT1 Indicates that a PR has LGTM 1. type/bugfix This PR fixes a bug. type/4.0-cherry-pick
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants