Add backup manager tool #694
Conversation
| BackupScheduleLabelKey string = "tidb.pingcap.com/backup-schedule" | ||
|
|
||
| // BackupProtectionFinalizer is the name of finalizer on backups | ||
| BackupProtectionFinalizer string = "tidb.pingcap.com/backup-protection" |
There was a problem hiding this comment.
where are those constants used?
There was a problem hiding this comment.
Will be used in backup controller.
| "--verbose=3", | ||
| } | ||
|
|
||
| dumper := exec.Command("/mydumper", args...) |
There was a problem hiding this comment.
why not use exec.Command().CombinedOutput(), then you can get the output when failed.
There was a problem hiding this comment.
On the one hand, it is divided into multiple steps to make the error finer. On the other hand, it is possible to introduce streaming backup data in the future. This requires exporting the stdout of the command, refer to this.
Co-Authored-By: weekface <weekface@gmail.com>
Co-Authored-By: weekface <weekface@gmail.com>
Makefile
Outdated
| @@ -45,6 +45,12 @@ discovery: | |||
| admission-controller: | |||
| $(GO) -ldflags '$(LDFLAGS)' -o images/tidb-operator/bin/tidb-admission-controller cmd/admission-controller/main.go | |||
|
|
|||
| backup-manager: | |||
| $(GO) -ldflags '$(LDFLAGS)' -o images/backup-manager/bin/backup-manager cmd/backup-manager/main.go | |||
There was a problem hiding this comment.
should we add tidb- prefix too? see tidb-admission-controller
|
|
||
| func (bo *BackupOpts) getTikvGCLifeTime(db *sql.DB) (string, error) { | ||
| var tikvGCTime string | ||
| row := db.QueryRow("select variable_value from ? where variable_name= ?", constants.TidbMetaTable, constants.TikvGCVariable) |
There was a problem hiding this comment.
This has SQL injection problem.
There was a problem hiding this comment.
On the one hand, the input parameters of this SQL are controllable. In addition, parameterized statements use parameters instead of concatenating input variables in embedded SQL statements is safe. Refer to here
| } | ||
|
|
||
| func (bo *BackupOpts) setTikvGClifeTime(db *sql.DB, gcTime string) error { | ||
| _, err := db.Exec("update ? set variable_value = ? where variable_name = ?", constants.TidbMetaTable, gcTime, constants.TikvGCVariable) |
| return nil | ||
| } | ||
|
|
||
| func (bo *BackupOpts) cleanRemoteBackupData(bucket string) error { |
There was a problem hiding this comment.
do we need to clean local backup data?
There was a problem hiding this comment.
I will add the cleanup logic of local backup data after the manual test passes
| return bm.StatusUpdater.Update(backup, &v1alpha1.BackupCondition{ | ||
| Type: v1alpha1.BackupFailed, | ||
| Status: corev1.ConditionTrue, | ||
| Reason: "GetBackupSizeFailed", |
There was a problem hiding this comment.
I will handle it in the next PR
|
/run-e2e-tests |
1 similar comment
|
/run-e2e-tests |
|
/run-e2e-tests |
|
/run-e2e-tests |
|
cherry pick to release-1.1 failed |
| && unzip rclone-${VERSION}-linux-amd64.zip \ | ||
| && mv rclone-${VERSION}-linux-amd64/rclone /rclone \ | ||
| && chmod 755 /rclone \ | ||
| && rm -rf rclone-${VERSION}-linux-amd64.zip |
There was a problem hiding this comment.
need to rm rclone-${VERSION}-linux-amd64 directory as well
| $BACKUP_BIN $VERBOSE schedule-backup "$@" | ||
| ;; | ||
| *) | ||
| echo "Usage: $0 {backup|restore|clean}" |
There was a problem hiding this comment.
add schedule-backup in Usage
* .github: add external link check * align pingcap/docs-dm#370 * update regex to exclude links * fix dead external links Signed-off-by: Ran <huangran@pingcap.com> Co-authored-by: Ran <huangran@pingcap.com>
What problem does this PR solve?
What is changed and how does it work?
This tool is used by backup and restore controller
Check List
Tests
Code changes
Side effects
Related changes
Does this PR introduce a user-facing change?: