br/restore: add version check for backup schema_version #929

YuJuncen · 2021-03-25T03:37:56Z

What problem does this PR solve?

a special fix for v4.0 versions of #790

Because #685 cannot be cherry-picked to release-4.0, 4.0.x BR would corrupt 4.0 clusters without even a warn message when the backup archive come from a table with non-int clustered primary key. This PR would add a check to the backup archive version, and intercept restoring from incompatible backup archives.

What is changed and how it works?

CheckClusterVersion was changed to check version in different way.
And added a check to cluster version stored in backupmeta during restore. (after reading backup meta)

Currently, this check only checks major versions (because of clustered index).

Check List

Tests

Unit test
Manual test (add detailed scripts or steps below)

Code changes

Has exported function/method change
signature change for CheckClusterVersion.

- CheckClusterVersion(ctx context.Context, client pd.Client) error
+ CheckClusterVersion(ctx context.Context, client pd.Client, checker VerChecker) error

Release Note

BR would check cluster version of backup now.

YuJuncen · 2021-03-25T05:51:51Z

/run-all-tests

YuJuncen · 2021-03-25T07:37:47Z

/build

overvenus

rest LGTM

pkg/version/version.go

overvenus · 2021-04-21T05:41:55Z

pkg/version/version.go

@@ -199,3 +224,20 @@ func CheckTiDBVersion(versionStr string, requiredMinVersion, requiredMaxVersion
 	}
 	return CheckVersion("TiDB", *version, requiredMinVersion, requiredMaxVersion)
 }
+
+// NormalizeBackupVersion normalizes the version string from backupmeta.
+func NormalizeBackupVersion(version string) *semver.Version {


Seem version.removeVAndHash can be replaced by this funcation.

You mean merge those two functions? Seems removeVAndHash focus on the version string made by the make script, and NormalizeBackupVersion focus on the version string returned from PD. Maybe it wouldn't work if we just replace the former with the latter. 🤔

Co-authored-by: Neil Shen <overvenus@gmail.com>

YuJuncen · 2021-04-23T02:51:14Z

/run-integration-tests

port in use

pkg/version/version.go

YuJuncen · 2021-04-30T02:35:59Z

listen tcp 0.0.0.0:10080: bind: address already in use

/run-integration-tests

overvenus · 2021-04-30T02:49:12Z

/lgtm

3pointer

/lgtm

ti-chi-bot · 2021-05-08T11:10:03Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

3pointer
overvenus

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by writing /lgtm in a comment.
Reviewer can cancel approval by writing /lgtm cancel in a comment.

3pointer · 2021-05-08T11:10:19Z

/merge

ti-chi-bot · 2021-05-08T11:10:21Z

This pull request has been accepted and is ready to merge.

Commit hash: c568f98

YuJuncen · 2021-05-09T12:11:30Z

[2021-05-08T11:29:42.533Z] [2021/05/08 19:29:40.505 +08:00] [INFO] [restore_raw.go:110] ["all files are filtered out from the backup archive, nothing to restore"]

/run-integration-tests

3pointer · 2021-05-10T06:24:43Z

/run-integration-tests

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot · 2021-05-10T06:53:22Z

In response to a cherrypick label: new pull request created: #1090.

ti-chi-bot · 2021-05-10T06:53:58Z

In response to a cherrypick label: new pull request created: #1091.

YuJuncen added 3 commits March 25, 2021 10:53

version, restore: add version check.

e79699f

restore: use cluster version

c544aa7

*: fix lint

48c7445

YuJuncen added needs-cherry-pick-release-4.0 component/restore needs-cherry-pick-release-5.0 labels Mar 25, 2021

ti-chi-bot added the size/L label Mar 25, 2021

ti-chi-bot requested a review from 3pointer March 25, 2021 03:38

YuJuncen and others added 3 commits March 25, 2021 11:43

version: trim backup version anyway

d5a52bb

restore: rename some variables

06b9a74

Merge branch 'master' into ver-check

4b661dc

YuJuncen requested review from overvenus and kennytm March 25, 2021 05:03

YuJuncen added 4 commits March 26, 2021 10:58

Merge branch 'master' into ver-check

a72d4e7

Merge branch 'master' into ver-check

f68049b

Merge branch 'master' into ver-check

498a30c

Merge branch 'master' into ver-check

657a69b

YuJuncen added the status/ptal label Mar 31, 2021

YuJuncen added this to the v4.0.13 milestone Apr 1, 2021

overvenus reviewed Apr 21, 2021

View reviewed changes

YuJuncen and others added 3 commits April 22, 2021 17:26

Merge branch 'master' into ver-check

5ce6d95

Apply suggestions from code review

13e18a4

Co-authored-by: Neil Shen <overvenus@gmail.com>

utils: run go fmt

462549b

3pointer reviewed Apr 28, 2021

View reviewed changes

pkg/version/version.go Outdated Show resolved Hide resolved

YuJuncen added 2 commits April 28, 2021 11:57

Merge branch 'master' of https://github.com/pingcap/br into ver-check

4e224f5

version: address comment

f14958b

YuJuncen requested a review from overvenus April 28, 2021 06:27

YuJuncen requested a review from 3pointer April 28, 2021 06:27

Merge branch 'master' into ver-check

6e8eede

ti-chi-bot added the status/LGT1 LGTM1 label Apr 30, 2021

YuJuncen mentioned this pull request Apr 30, 2021

br: add restore to systables pingcap/docs-cn#6194

Merged

11 tasks

Merge branch 'master' into ver-check

c568f98

3pointer reviewed May 8, 2021

View reviewed changes

ti-chi-bot added status/LGT2 LGTM2 and removed status/LGT1 LGTM1 labels May 8, 2021

ti-chi-bot added the status/can-merge label May 8, 2021

Merge branch 'master' into ver-check

9215fb9

ti-chi-bot merged commit 70b0a35 into pingcap:master May 10, 2021

ti-chi-bot pushed a commit to ti-chi-bot/br that referenced this pull request May 10, 2021

This is an automated cherry-pick of pingcap#929

690d222

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot mentioned this pull request May 10, 2021

br/restore: add version check for backup schema_version (#929) #1090

Merged

ti-chi-bot mentioned this pull request May 10, 2021

br/restore: add version check for backup schema_version (#929) #1091

Merged

ti-chi-bot added a commit that referenced this pull request May 11, 2021

br/restore: add version check for backup schema_version (#929) (#1090)

40c1967

ti-chi-bot added a commit that referenced this pull request May 26, 2021

br/restore: add version check for backup schema_version (#929) (#1091)

719285d

3pointer pushed a commit to 3pointer/br that referenced this pull request Jun 4, 2021

br/restore: add version check for backup schema_version (pingcap#929)

b4fb6ae

3pointer pushed a commit to 3pointer/br that referenced this pull request Jun 4, 2021

br/restore: add version check for backup schema_version (pingcap#929)

dd99592

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

br/restore: add version check for backup schema_version #929

br/restore: add version check for backup schema_version #929

YuJuncen commented Mar 25, 2021 •

edited

Loading

YuJuncen commented Mar 25, 2021

YuJuncen commented Mar 25, 2021

overvenus left a comment

overvenus Apr 21, 2021

YuJuncen Apr 22, 2021 •

edited

Loading

YuJuncen commented Apr 23, 2021

YuJuncen commented Apr 30, 2021

overvenus commented Apr 30, 2021

3pointer left a comment

ti-chi-bot commented May 8, 2021

3pointer commented May 8, 2021

ti-chi-bot commented May 8, 2021

YuJuncen commented May 9, 2021

3pointer commented May 10, 2021

ti-chi-bot commented May 10, 2021

ti-chi-bot commented May 10, 2021

br/restore: add version check for backup schema_version #929

br/restore: add version check for backup schema_version #929

Conversation

YuJuncen commented Mar 25, 2021 • edited Loading

What problem does this PR solve?

What is changed and how it works?

Check List

Release Note

YuJuncen commented Mar 25, 2021

YuJuncen commented Mar 25, 2021

overvenus left a comment

Choose a reason for hiding this comment

overvenus Apr 21, 2021

Choose a reason for hiding this comment

YuJuncen Apr 22, 2021 • edited Loading

Choose a reason for hiding this comment

YuJuncen commented Apr 23, 2021

YuJuncen commented Apr 30, 2021

overvenus commented Apr 30, 2021

3pointer left a comment

Choose a reason for hiding this comment

ti-chi-bot commented May 8, 2021

3pointer commented May 8, 2021

ti-chi-bot commented May 8, 2021

YuJuncen commented May 9, 2021

3pointer commented May 10, 2021

ti-chi-bot commented May 10, 2021

ti-chi-bot commented May 10, 2021

YuJuncen commented Mar 25, 2021 •

edited

Loading

YuJuncen Apr 22, 2021 •

edited

Loading