Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ticdc: add new rules and information for syncpoint #17710

Merged
merged 7 commits into from
Jun 27, 2024
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion ticdc/ticdc-upstream-downstream-check.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,12 @@ select * from tidb_cdc.syncpoint_v1;

## 注意事项

- TiCDC 在创建 Changefeed 前,请确保 TiCDC 的配置项 `enable-sync-point` 已设置为 `true`,这样才会开启 Syncpoint 功能,在下游保存 `ts-map`。完整的配置请参考 [TiCDC 同步任务配置文件描述](/ticdc/ticdc-changefeed-config.md)。
- TiCDC 在创建 Changefeed 前,请确保 TiCDC 的配置项 `enable-sync-point` 已设置为 `true`,这样才会开启 Syncpoint 功能,在下游保存 `ts-map`。配置项 `sync-point-interval` 默认格式为 h m s, 例如 "1h30m30s",最小值支持 30s。更多完整的配置信息请参考 [TiCDC 同步任务配置文件描述](/ticdc/ticdc-changefeed-config.md)。
hongyunyan marked this conversation as resolved.
Show resolved Hide resolved
- 在使用 Syncpoint 功能进行数据校验时,需要调整 TiKV 的 GC 时间,保证在校验时 snapshot 对应的历史数据不会被执行 GC。建议调整为 1 个小时,在校验后再还原 GC 设置。
- 以上配置只展示了 `Datasource config` 部分,完整配置请参考 [sync-diff-inspector 用户文档](/sync-diff-inspector/sync-diff-inspector-overview.md)。
- 从 v6.4.0 开始,TiCDC 使用 Syncpoint 功能需要同步任务拥有下游集群的 `SYSTEM_VARIABLES_ADMIN` 或者 `SUPER` 权限。
- 从 v8.2.0 开始,TiCDC 对 primary_ts 值的生成规则做了以下调整:
- 每当 TiCDC 产生一个新的 primary_ts 时,它必须是 `sync-point-interval` 的整数倍。
- 对于每个新的 changefeed,TiCDC 会计算出一个初始的 primary_ts。这个初始值是大于或等于 changefeed 开始时间(startTs)的最小的 `sync-point-interval` 的整数倍。

该设定保证在数据同步过程中,多个下游中 primary_ts 时间戳的一致性和可预测性,便于各个下游集群中进行数据的同步和恢复。
hongyunyan marked this conversation as resolved.
Show resolved Hide resolved