Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update log backup user docs #2641

Open
wants to merge 25 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions en/backup-restore-cr.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,13 @@ This section introduces the fields in the `Backup` CR.
* `volume-snapshot`: back up data by volume snapshots.
* `log`: back up log data in real time in the KV layer.

* `.spec.logSubcommand`: the subcommand for controlling the log backup status in the Backup CR. This field provides three options for managing a log backup task:
RidRisR marked this conversation as resolved.
Show resolved Hide resolved
* `log-start`: Initiates a new log backup task or resumes an existing task that has been paused. Use this option to start the log backup process or to continue it from a paused state.
* `log-pause`: Temporarily pauses an active log backup task, allowing it to be resumed later using the log-start command.
* `log-stop`: Permanently stops the log backup task. When this command is issued, the Backup CR enters a stopped state and cannot be restarted.

Note that in versions earlier than v1.5.5, this field does not exist. you could use the `logStop`: true/false field to stop or start a task. In the v1.5.5+ and v1.6.1+ verson, the `logStop` is still aviliable but not recommanded.

* `.spec.restoreMode`: the restore mode. The default value is `snapshot`, which means restoring data from snapshots in the KV layer. This field is valid only for restore and has three value options currently:
* `snapshot`: restore data from snapshots in the KV layer.
* `volume-snapshot`: restore data from volume snapshots.
Expand Down
2 changes: 2 additions & 0 deletions en/backup-restore-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,8 @@ kubectl delete backupschedule ${name} -n ${namespace}

If you use TiDB Operator v1.1.2 or an earlier version, or if you use TiDB Operator v1.1.3 or a later version and set the value of `spec.cleanPolicy` to `Delete`, TiDB Operator cleans the backup data when it deletes the CR.

If you are using version 1.5.5+ or 1.6.1+, when you delete a CR, TiDB Operator will attempt to automatically stop the running log backup task. The auto-stop feature will only stop log backup tasks that are running on track; it does not handle tasks that are in an error or failed state.

If you back up cluster data using AWS EBS volume snapshots and set the value of `spec.cleanPolicy` to `Delete`, TiDB Operator deletes the CR, and cleans up the backup file and the volume snapshots on AWS.

In such cases, if you need to delete the namespace, it is recommended that you first delete all the `Backup`/`BackupSchedule` CRs and then delete the namespace.
Expand Down
118 changes: 113 additions & 5 deletions en/backup-to-aws-s3-using-br.md
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,24 @@ demo1-full-backup-s3 full snapshot Complete s3://my-bucket/my-full-backu

You can use a `Backup` CR to describe the start and stop of a log backup task and manage the log backup data. Log backup grants permissions to remote storages in the same way as snapshot backup. In this section, the example shows log backup operations by taking a `Backup` CR named `demo1-log-backup-s3` as an example. Note that these operations assume that permissions to remote storages are granted using accessKey and secretKey. See the following detailed steps.

#### Log backup subcommands
csuzhangxc marked this conversation as resolved.
Show resolved Hide resolved

The logSubcommand field in the Backup CR allows you to control the status of a log backup task. There are three valid inputs for logSubcommand:

- log-start: This command initiates a new log backup task or resumes an existing task that has been paused. It can be used to start the log backup process or resume from the paused state.

- log-pause: This command temporarily pauses an active log backup task. The task can be resumed later using the log-start command.

- log-stop: This command permanently stops the log backup task. When this command is issued, the Backup CR enters a stopped state and cannot be restarted.

These commands allow fine-grained control over the lifecycle of log backup tasks, enabling start, pause, resume, and stop operations to manage log data retention in a Kubernetes environment.

<Tip>
In TiDB Operator versions earlier than v1.5.4/v1.6.0, you could use the logStop: true/false field to stop or start a task. This field is retained for backward compatibility.

However, you must not mix logStop with logSubcommand in the same Backup CR. Doing so is unsupported, and using logStop is not recommended in later versions. Stick to logSubcommand for better clarity and consistency.
</Tip>

#### Start log backup

1. In the `backup-test` namespace, create a `Backup` CR named `demo1-log-backup-s3`.
Expand All @@ -247,6 +265,7 @@ You can use a `Backup` CR to describe the start and stop of a log backup task an
namespace: backup-test
spec:
backupMode: log
logSubcommand: log-start
br:
cluster: demo1
clusterNamespace: test1
Expand Down Expand Up @@ -308,15 +327,101 @@ Conditions:
Log Checkpoint Ts: 436569119308644661
```

#### Pause log backup

Because you already created a `Backup` CR named `demo1-log-backup-s3` when you started log backup, you can pause the log backup by modifying the same `Backup` CR.

```shell
kubectl edit backup demo1-log-backup-s3 -n backup-test
```

To pause the log backup task, you only need to change the `logSubcommand` from `log-start` to `log-pause`. Then save and quit the editor. The modified content is as follows:

```yaml
---
apiVersion: pingcap.com/v1alpha1
kind: Backup
metadata:
name: demo1-log-backup-s3
namespace: backup-test
spec:
backupMode: log
logSubcommand: log-pause
br:
cluster: demo1
clusterNamespace: test1
sendCredToTikv: true
s3:
provider: aws
secretName: s3-secret
region: us-west-1
bucket: my-bucket
prefix: my-log-backup-folder
```

You can see the `STATUS` of the `Backup` CR named `demo1-log-backup-s3` change from `Running` to `Pause`:

```shell
kubectl get backup -n backup-test
```

```
NAME MODE STATUS ....
demo1-log-backup-s3 log Pause ....
```

#### Resume log backup

If a log backup task is paused, you could set `logSubcommand: log-start` to resume it. Be aware that, you couldn't resume a task from `Fail` or `Stopped` state.

```shell
kubectl edit backup demo1-log-backup-s3 -n backup-test
```

To resume the log backup task, you only need to change the `logSubcommand` from `log-pause` to `log-start`. Then save and quit the editor. The modified content is as follows:

```yaml
---
apiVersion: pingcap.com/v1alpha1
kind: Backup
metadata:
name: demo1-log-backup-s3
namespace: backup-test
spec:
backupMode: log
logSubcommand: log-start
br:
cluster: demo1
clusterNamespace: test1
sendCredToTikv: true
s3:
provider: aws
secretName: s3-secret
region: us-west-1
bucket: my-bucket
prefix: my-log-backup-folder
```

You can see the `STATUS` of the `Backup` CR named `demo1-log-backup-s3` change from `Pause` to `Running`:

```shell
kubectl get backup -n backup-test
```

```
NAME MODE STATUS ....
demo1-log-backup-s3 log Running ....
```

#### Stop log backup

Because you already created a `Backup` CR named `demo1-log-backup-s3` when you started log backup, you can stop the log backup by modifying the same `Backup` CR. The priority of all operations is: stop log backup > delete log backup data > start log backup.
Because you already created a `Backup` CR named `demo1-log-backup-s3` when you started log backup, you can stop the log backup by modifying the same `Backup` CR.

```shell
kubectl edit backup demo1-log-backup-s3 -n backup-test
```

In the last line of the CR, append `spec.logStop: true`. Then save and quit the editor. The modified content is as follows:
Change the `logSubcommand` to `log-stop`. Then save and quit the editor. The modified content is as follows:

```yaml
---
Expand All @@ -327,6 +432,7 @@ metadata:
namespace: backup-test
spec:
backupMode: log
logSubcommand: log-stop
br:
cluster: demo1
clusterNamespace: test1
Expand All @@ -337,7 +443,6 @@ spec:
region: us-west-1
bucket: my-bucket
prefix: my-log-backup-folder
logStop: true
```

You can see the `STATUS` of the `Backup` CR named `demo1-log-backup-s3` change from `Running` to `Stopped`:
Expand All @@ -352,12 +457,14 @@ demo1-log-backup-s3 log Stopped ....
```

<Tip>
You can also stop log backup by taking the same steps as in [Start log backup](#start-log-backup). The existing `Backup` CR will be updated.
Stopped is the terminated state of a log backup CR, you couldn't change the state again, but you still could clean log backup data.

In TiDB Operator versions earlier than v1.5.4/v1.6.0, you could use the logStop: true/false field to stop or start a task. This field is retained for backward compatibility.
</Tip>

#### Clean log backup data

1. Because you already created a `Backup` CR named `demo1-log-backup-s3` when you started log backup, you can clean the log data backup by modifying the same `Backup` CR. The priority of all operations is: stop log backup > delete log backup data > start log backup. The following example shows how to clean log backup data generated before 2022-10-10T15:21:00+08:00.
1. Because you already created a `Backup` CR named `demo1-log-backup-s3` when you started log backup, you can clean the log data backup by modifying the same `Backup` CR. The following example shows how to clean log backup data generated before 2022-10-10T15:21:00+08:00.

```shell
kubectl edit backup demo1-log-backup-s3 -n backup-test
Expand All @@ -374,6 +481,7 @@ You can also stop log backup by taking the same steps as in [Start log backup](#
namespace: backup-test
spec:
backupMode: log
logSubcommand: log-start/log-pause/log-stop
br:
cluster: demo1
clusterNamespace: test1
Expand Down
116 changes: 111 additions & 5 deletions en/backup-to-azblob-using-br.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,24 @@ demo1-full-backup-azblob full snapshot Complete azure://my-container/my-

You can use a `Backup` CR to describe the start and stop of a log backup task and manage the log backup data. In this section, the example shows how to create a `Backup` CR named `demo1-log-backup-azblob`. See the following detailed steps.

#### Log backup subcommands

The logSubcommand field in the Backup CR allows you to control the status of a log backup task. There are three valid inputs for logSubcommand:

- log-start: This command initiates a new log backup task or resumes an existing task that has been paused. It can be used to start the log backup process or resume from the paused state.

- log-pause: This command temporarily pauses an active log backup task. The task can be resumed later using the log-start command.

- log-stop: This command permanently stops the log backup task. When this command is issued, the Backup CR enters a stopped state and cannot be restarted.

These commands allow fine-grained control over the lifecycle of log backup tasks, enabling start, pause, resume, and stop operations to manage log data retention in a Kubernetes environment.

<Tip>
In TiDB Operator versions earlier than v1.5.4/v1.6.0, you could use the logStop: true/false field to stop or start a task. This field is retained for backward compatibility.

However, you must not mix logStop with logSubcommand in the same Backup CR. Doing so is unsupported, and using logStop is not recommended in later versions. Stick to logSubcommand for better clarity and consistency.
</Tip>

#### Start log backup

1. In the `backup-test` namespace, create a `Backup` CR named `demo1-log-backup-azblob`.
Expand All @@ -158,6 +176,7 @@ You can use a `Backup` CR to describe the start and stop of a log backup task an
namespace: backup-test
spec:
backupMode: log
logSubcommand: log-start
br:
cluster: demo1
clusterNamespace: test1
Expand Down Expand Up @@ -218,15 +237,99 @@ Conditions:
Log Checkpoint Ts: 436569119308644661
```

#### Pause log backup

Because you already created a `Backup` CR named `demo1-log-backup-azblob` when you started log backup, you can pause the log backup by modifying the same `Backup` CR.

```shell
kubectl edit backup log-backup-azblob -n backup-test
```

To pause the log backup task, you only need to change the `logSubcommand` from `log-start` to `log-pause`. Then save and quit the editor. The modified content is as follows:

```yaml
---
apiVersion: pingcap.com/v1alpha1
kind: Backup
metadata:
name: demo1-log-backup-azblob
namespace: backup-test
spec:
backupMode: log
logSubcommand: log-pause
br:
cluster: demo1
clusterNamespace: test1
sendCredToTikv: true
azblob:
secretName: azblob-secret
container: my-container
prefix: my-log-backup-folder
#accessTier: Hot
```

You can see the `STATUS` of the `Backup` CR named `log-backup-azblob` change from `Running` to `Pause`:

```shell
kubectl get backup -n backup-test
```

```
NAME MODE STATUS ....
log-backup-azblob log Pause ....
```

#### Resume log backup

If a log backup task is paused, you could set `logSubcommand: log-start` to resume it. Be aware that, you couldn't resume a task from `Fail` or `Stopped` state.

```shell
kubectl edit backup log-backup-azblob.yaml -n backup-test
```

To resume the log backup task, you only need to change the `logSubcommand` from `log-pause` to `log-start`. Then save and quit the editor. The modified content is as follows:

```yaml
---
apiVersion: pingcap.com/v1alpha1
kind: Backup
metadata:
name: demo1-log-backup-azblob
namespace: backup-test
spec:
backupMode: log
logSubcommand: log-start
br:
cluster: demo1
clusterNamespace: test1
sendCredToTikv: true
azblob:
secretName: azblob-secret
container: my-container
prefix: my-log-backup-folder
#accessTier: Hot
```

You can see the `STATUS` of the `Backup` CR named `log-backup-azblob` change from `Pause` to `Running`:

```shell
kubectl get backup -n backup-test
```

```
NAME MODE STATUS ....
log-backup-azblob log Running ....
```

#### Stop log backup

Because you already created a `Backup` CR named `demo1-log-backup-azblob` when you started log backup, you can stop the log backup by modifying the same `Backup` CR. The priority of all operations is: stop log backup > delete log backup data > start log backup.
Because you already created a `Backup` CR named `demo1-log-backup-azblob` when you started log backup, you can stop the log backup by modifying the same `Backup` CR.

```shell
kubectl edit backup demo1-log-backup-azblob -n backup-test
```

In the last line of the CR, append `spec.logStop: true`. Then save and quit the editor. The modified content is as follows:
Change the `logSubcommand` to `log-stop`. Then save and quit the editor. The modified content is as follows:

```yaml
---
Expand All @@ -237,6 +340,7 @@ metadata:
namespace: backup-test
spec:
backupMode: log
logSubcommand: log-stop
br:
cluster: demo1
clusterNamespace: test1
Expand All @@ -246,7 +350,6 @@ spec:
container: my-container
prefix: my-log-backup-folder
#accessTier: Hot
logStop: true
```

You can see the `STATUS` of the `Backup` CR named `demo1-log-backup-azblob` change from `Running` to `Stopped`:
Expand All @@ -261,12 +364,14 @@ demo1-log-backup-azblob log Stopped ....
```

<Tip>
You can stop log backup by taking the same steps as in [Start log backup](#start-log-backup). The existing `Backup` CR will be updated.
Stopped is the terminated state of a log backup CR, you couldn't change the state again, but you still could clean log backup data.

In TiDB Operator versions earlier than v1.5.4/v1.6.0, you could use the logStop: true/false field to stop or start a task. This field is retained for backward compatibility.
</Tip>

#### Clean log backup data

1. Because you already created a `Backup` CR named `demo1-log-backup-azblob` when you started log backup, you can clean the log data backup by modifying the same `Backup` CR. The priority of all operations is: stop log backup > delete log backup data > start log backup. The following example shows how to clean log backup data generated before 2022-10-10T15:21:00+08:00.
1. Because you already created a `Backup` CR named `demo1-log-backup-azblob` when you started log backup, you can clean the log data backup by modifying the same `Backup` CR. The following example shows how to clean log backup data generated before 2022-10-10T15:21:00+08:00.

```shell
kubectl edit backup demo1-log-backup-azblob -n backup-test
Expand All @@ -283,6 +388,7 @@ You can stop log backup by taking the same steps as in [Start log backup](#start
namespace: backup-test
spec:
backupMode: log
logSubcommand: log-start/log-pause/log-stop
br:
cluster: demo1
clusterNamespace: test1
Expand Down
Loading
Loading