Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tikv: fix infinite retry when kv region continuing to return StaleCommand error (#16481) #16528

Merged
merged 6 commits into from
Apr 23, 2020

Conversation

sre-bot
Copy link
Contributor

@sre-bot sre-bot commented Apr 17, 2020

cherry-pick #16481 to release-3.0


What problem does this PR solve?

Issue Number: close #16524

Problem Summary:

tikv region will report staleCommand when it can not catch up the log and will let tidb do retry.

but stale error didn't use backoff util, so it will infinite loop when some kv region keep return stale.

it can be reproduced by TestOnRegionError

this will make SQLs block forever if it hit those regions.

What is changed and how it works?

What's Changed:

do a little backoff before retry stale command,

How it Works:

do backoff, let retry loop has chances be broken by maxSleepTime or kill command

Related changes

  • Need to cherry-pick to the release branch(3.x need modify parser)

Check List

Tests

  • Unit test

Side effects

  • n/a

Release note

fix infinite retry when kv continuing to return staleCommand error


This change is Reviewable

@sre-bot
Copy link
Contributor Author

sre-bot commented Apr 17, 2020

/run-all-tests

@sre-bot
Copy link
Contributor Author

sre-bot commented Apr 19, 2020

@coocood, @tiancaiamao, @jackysp, @crazycs520, PTAL.

@lysu
Copy link
Contributor

lysu commented Apr 20, 2020

need merge pingcap/parser#815 first

@lysu
Copy link
Contributor

lysu commented Apr 20, 2020

/build

@sre-bot
Copy link
Contributor Author

sre-bot commented Apr 22, 2020

@coocood, @tiancaiamao, @jackysp, @crazycs520, PTAL.

@jackysp
Copy link
Member

jackysp commented Apr 22, 2020

need merge pingcap/parser#815 first

It's merged.

Copy link
Member

@zz-jason zz-jason left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zz-jason zz-jason added the status/LGT1 Indicates that a PR has LGTM 1. label Apr 22, 2020
@coocood
Copy link
Member

coocood commented Apr 23, 2020

@lysu
It seems that the two added files are not used.

@lysu
Copy link
Contributor

lysu commented Apr 23, 2020

@coocood may fault, rebase seems didn't report conflict for those two files, removed~

Copy link
Member

@coocood coocood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@coocood
Copy link
Member

coocood commented Apr 23, 2020

/merge

@sre-bot sre-bot added the status/can-merge Indicates a PR has been approved by a committer. label Apr 23, 2020
@sre-bot
Copy link
Contributor Author

sre-bot commented Apr 23, 2020

/run-all-tests

@sre-bot sre-bot merged commit 445e69b into pingcap:release-3.0 Apr 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/tikv status/can-merge Indicates a PR has been approved by a committer. status/LGT1 Indicates that a PR has LGTM 1. type/bugfix This PR fixes a bug. type/3.0-cherry-pick
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants