Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ddl: use kv.Key as reorg progress marker instead of kv.Handle #20908

Merged
merged 8 commits into from
Nov 11, 2020

Conversation

tangenta
Copy link
Contributor

@tangenta tangenta commented Nov 6, 2020

What problem does this PR solve?

Issue Number: close #20727

Problem Summary:

For now, ADD INDEX(as well as some other DDL jobs that need data-reorganization) assume that the 'startKey' and 'endKey' of a region can always be decoded as handle out:

func decodeHandleRange(keyRange kv.KeyRange) (kv.Handle, kv.Handle, error) {

What is changed and how it works?

What's Changed:

The reason why TiDB decodes keys to handles during 'ADD INDEX' is that the reorganization progress is persisted in storage. The handles are used as the progress marker. This PR change the type of marker to kv.Key.

How it Works:

Using `kv.Key as the marker skips the decoding routine.

Related changes

N/A

Check List

Tests

  • Unit test
  • Integration test

Side effects

  • Compatibility: this PR should not break compatibility because the binary representation of the progress marker is unchanged.
  • Performance: to improve human readability, the key is need to be decoded every time TiDB logs.

Release note

  • No release note

@tangenta tangenta added the sig/sql-infra SIG: SQL Infra label Nov 6, 2020
@tangenta tangenta requested a review from a team as a code owner November 6, 2020 09:42
@tangenta tangenta requested review from XuHuaiyu and removed request for a team November 6, 2020 09:42
@github-actions github-actions bot added the sig/execution SIG execution label Nov 6, 2020
ddl/backfilling.go Outdated Show resolved Hide resolved
Copy link
Contributor

@AilinKid AilinKid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rest LGTM

ddl/reorg.go Outdated Show resolved Hide resolved
ddl/reorg.go Outdated Show resolved Hide resolved
ddl/index.go Outdated Show resolved Hide resolved
ddl/column.go Outdated Show resolved Hide resolved
Copy link
Contributor

@AilinKid AilinKid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Nov 10, 2020
Copy link
Member

@wjhuang2016 wjhuang2016 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Nov 10, 2020
@tangenta
Copy link
Contributor Author

/merge

@ti-srebot ti-srebot added the status/can-merge Indicates a PR has been approved by a committer. label Nov 11, 2020
@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot ti-srebot merged commit d184120 into pingcap:master Nov 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/execution SIG execution sig/sql-infra SIG: SQL Infra status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

*: insufficient bytes decode value reported when adding index for some split common-handled tables.
4 participants