Skip to content
This repository has been archived by the owner on Aug 21, 2023. It is now read-only.

export: don't +1 TSO when set GC safepoint #290

Merged
merged 3 commits into from
Jul 6, 2021
Merged

export: don't +1 TSO when set GC safepoint #290

merged 3 commits into from
Jul 6, 2021

Conversation

lance6716
Copy link
Collaborator

@lance6716 lance6716 commented Jun 3, 2021 •
edited
Loading

What problem does this PR solve?

this error is hard to reproduce, so I added a 10 minutes sleep before any dump indeed occurs. The error may be slightly different but I think the root cause is same . Here's the log

[2021/06/03 19:20:30.262 +08:00] [INFO] [gc_worker.go:1531] ["[gc worker] sent safe point to PD"] [uuid=5e7451a0ff80001] ["safe point"=425385803120640001]
...
[2021/06/03 19:22:21.578 +08:00] [INFO] [set.go:312] ["load snapshot info schema"] [conn=5813] [SnapshotTS=425385803120640000]
...
[2021/06/03 19:22:21.579 +08:00] [INFO] [conn.go:812] ["command dispatched failed"] [conn=5813] [connInfo="id:5813, addr:127.0.0.1:62687 status:10, collation:utf8mb4_general_ci, user:root"] [command=Query] [status="inTxn:0, autocommit:1"] [sql="SET tidb_snapshot='2021-06-03 19:01:00'"] [txn_mode=PESSIMISTIC] [err="[tikv:9006]GC life time is shorter than transaction duration, transaction starts at 2021-06-03 19:01:00 +0800 CST, GC safe point is 2021-06-03 19:01:00 +0800 CST\ngithub.com/pingcap/errors.AddStack\n\t/Users/pingcap/gopkg/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20201126102027-b0a155152ca3/errors.go:174\ngithub.com/pingcap/errors.(*Error).GenWithStackByArgs\n\t/Users/pingcap/gopkg/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20201126102027-b0a155152ca3/normalize.go:156\ngithub.com/pingcap/tidb/store/tikv.(*KVStore).CheckVisibility\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/store/tikv/kv.go:104\ngithub.com/pingcap/tidb/store/tikv.(*tikvSnapshot).Get\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/store/tikv/snapshot.go:387\ngithub.com/pingcap/tidb/structure.(*TxStructure).Get\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/structure/string.go:36\ngithub.com/pingcap/tidb/structure.(*TxStructure).GetInt64\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/structure/string.go:45\ngithub.com/pingcap/tidb/meta.(*Meta).GetSchemaVersion\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/meta/meta.go:260\ngithub.com/pingcap/tidb/domain.(*Domain).loadInfoSchema\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/domain/domain.go:102\ngithub.com/pingcap/tidb/domain.(*Domain).GetSnapshotInfoSchema\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/domain/domain.go:316\ngithub.com/pingcap/tidb/executor.(*SetExecutor).loadSnapshotInfoSchemaIfNeeded\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/executor/set.go:314\ngithub.com/pingcap/tidb/executor.(*SetExecutor).setSysVariable\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/executor/set.go:195\ngithub.com/pingcap/tidb/executor.(*SetExecutor).Next\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/executor/set.go:108\ngithub.com/pingcap/tidb/executor.Next\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/executor/executor.go:277\ngithub.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelayExecutor\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/executor/adapter.go:537\ngithub.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelay\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/executor/adapter.go:418\ngithub.com/pingcap/tidb/executor.(*ExecStmt).Exec\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/executor/adapter.go:368\ngithub.com/pingcap/tidb/session.runStmt\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/session/session.go:1531\ngithub.com/pingcap/tidb/session.(*session).ExecuteStmt\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/session/session.go:1426\ngithub.com/pingcap/tidb/server.(*TiDBContext).ExecuteStmt\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/server/driver_tidb.go:218\ngithub.com/pingcap/tidb/server.(*clientConn).handleStmt\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/server/conn.go:1630\ngithub.com/pingcap/tidb/server.(*clientConn).handleQuery\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/server/conn.go:1503\ngithub.com/pingcap/tidb/server.(*clientConn).dispatch\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/server/conn.go:1037\ngithub.com/pingcap/tidb/server.(*clientConn).Run\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/server/conn.go:795\ngithub.com/pingcap/tidb/server.(*Server).onConn\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/server/server.go:477\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1357"]

What is changed and how it works?

as title

Check List

Tests

  • Manual test (TODO)

Side effects

Related changes

  • Need to cherry-pick to the release branch

Release note

@lance6716 lance6716 changed the title export: don't +1 when set GC safepoint export: don't +1 TSO when set GC safepoint Jun 3, 2021
lichunzhu
lichunzhu approved these changes Jun 3, 2021
View reviewed changes
Copy link
Contributor

@lichunzhu lichunzhu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot
Copy link
Member

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • lichunzhu

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added the status/LGT1 One reviewer approved (LGTM1) label Jun 3, 2021
@lichunzhu
Copy link
Contributor

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 2f7f052

lichunzhu
lichunzhu reviewed Jun 4, 2021
View reviewed changes
v4/export/sql.go Outdated Show resolved Hide resolved
@lance6716
Copy link
Collaborator Author

/hold

wait adding a test (but I'm investigating another issue, I prefer adding test later)

@glorv
Copy link
Collaborator

glorv commented Jun 4, 2021 •
edited
Loading

So Which versions does this issue actually affect?

@glorv
Copy link
Collaborator

glorv commented Jun 4, 2021

@lance6716 Please fix the unit test

@lichunzhu
Copy link
Contributor

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 4aac1e9

@lichunzhu
Copy link
Contributor

/merge

@lance6716
Copy link
Collaborator Author

/unhold

@ti-chi-bot ti-chi-bot merged commit 9aa01fb into master Jul 6, 2021
@ti-chi-bot ti-chi-bot deleted the lance6716-patch-1 branch July 6, 2021 09:27
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #302.

This was referenced Jul 6, 2021
Merged
Merged
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #303.

ti-chi-bot added a commit that referenced this pull request Jul 7, 2021
@ti-chi-bot
export: don't +1 TSO when set GC safepoint (#290) (#302)
f27040d
@ti-chi-bot ti-chi-bot mentioned this pull request Jul 23, 2021
Merged
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #316.

ti-chi-bot added a commit that referenced this pull request Jul 23, 2021
@ti-chi-bot
export: don't +1 TSO when set GC safepoint (#290) (#316)
41d0f04
ti-chi-bot added a commit that referenced this pull request Aug 20, 2021
@ti-chi-bot
export: don't +1 TSO when set GC safepoint (#290) (#303)
4051e78
tisonkun pushed a commit to tisonkun/dumpling that referenced this pull request Oct 20, 2021
@lance6716
export: don't +1 TSO when set GC safepoint (pingcap#290)
b299a5f
tisonkun pushed a commit to tisonkun/dumpling that referenced this pull request Oct 20, 2021
@lance6716
export: don't +1 TSO when set GC safepoint (pingcap#290)
3d6eba1
tisonkun pushed a commit to tisonkun/dumpling that referenced this pull request Oct 20, 2021
@lance6716
export: don't +1 TSO when set GC safepoint (pingcap#290)
3cef509
tisonkun pushed a commit to tisonkun/dumpling that referenced this pull request Oct 20, 2021
@lance6716
export: don't +1 TSO when set GC safepoint (pingcap#290)
af3504e
tisonkun pushed a commit to tisonkun/tidb that referenced this pull request Oct 20, 2021
@lance6716 @tisonkun
export: don't +1 TSO when set GC safepoint (pingcap/dumpling#290)
fdcffe8
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@lichunzhu lichunzhu lichunzhu approved these changes

@kennytm kennytm Awaiting requested review from kennytm

Assignees
No one assigned
Labels
cherry-pick-approved needs-cherry-pick-release-4.0 needs-cherry-pick-release-5.0 needs-cherry-pick-release-5.1 size/XS status/can-merge status/LGT1 One reviewer approved (LGTM1)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@lance6716 @ti-chi-bot @lichunzhu @glorv @nongfushanquan