-
Notifications
You must be signed in to change notification settings - Fork 5.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
server: retry executing sql without tiflash after tiflash is down #22459
server: retry executing sql without tiflash after tiflash is down #22459
Conversation
/run-all-tests |
/run-all-tests |
/cc @winoros |
/run-all-tests |
server/conn.go
Outdated
if err != nil { | ||
break | ||
if terror.ErrorEqual(err, tikv.ErrTiFlashServerTimeout) && retryable { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's better to introduce a switch(generally it is implemented as a SQL variable) to control if we retry here.
/run-all-tests |
/run-mybatis-test |
PTAL @winoros |
/run-all-tests |
/run-all-tests |
/run-all-tests |
/run-all-tests |
/run-all-tests |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
/merge |
Your auto merge job has been accepted, waiting for:
|
/run-all-tests |
What problem does this PR solve?
Problem Summary:
When we get a backoff error of TiFlash, the SQL will always fail while the TiKV nodes are actually alive. We hope to fallback to TiKV after TiFlash is down.
What is changed and how it works?
Proposal: https://docs.google.com/document/d/1dgq1MZQigTcZeck0iRLSCAu_ly3iGDr19Et6IHfQhCs/edit?usp=sharing (This PR implements simple solution section)
What's Changed & How it Works:
When running
clientConn.handleQuery
/clientConn.handleStmtExecute
, if we getErrTiFlashServerTimeout
error and the execution of SQL does't have side effect (If TiDB server already sends data to client or the execution is in cursor mode, the execution has side effect), we delete TiFlash fromIsolationReadEngines
, retry executing the SQL and add TiFlash back toIsolationReadEngines
. Besides, we use a switchtidb_enable_tiflash_fallback_tikv
to control whether to retry and the switch is off by default.Related changes
pingcap/docs
/pingcap/docs-cn
: system variable: add tidb_allow_fallback_to_tikv docs#4781 / system variable: add tidb_allow_fallback_to_tikv docs-cn#5484Check List
Tests
Release note