Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TiCDC: Add batch dmls mode. #7653

Closed
asddongmen opened this issue Nov 21, 2022 · 2 comments
Closed

TiCDC: Add batch dmls mode. #7653

asddongmen opened this issue Nov 21, 2022 · 2 comments
Labels
affects-6.1 area/ticdc Issues or PRs related to TiCDC. type/enhancement The issue or PR belongs to an enhancement.

Comments

@asddongmen
Copy link
Contributor

asddongmen commented Nov 21, 2022

Is your feature request related to a problem?

TiCDC always translates Delete and Update type row changes into a separate SQL statement and then synchronizes them downstream, for example, if you execute the following statement in the upstream TiDB:

Delete from `test`. `t1` where id < 100000;

Update `test`. `t1` set a = "whatever" where id > 100000 and id < 200000;

TiCDC will synchronize the row changes caused by the above Delete and Update statements one by one downstream, e.g.

delete from `test`. `t1` where id = 1 limit 1;

delete from `test`. `t1` where id = 2 limit 1;

delete from `test`. `t1` where id = 3 limit 1;
...
update `test`. `t1` set a = "whatever" where id = 1000001 limit 1;

update `test`. `t1` set a = "whatever" where id = 1000002 limit 1;

update `test`. `t1` set a = "whatever" where id = 1000003 limit 1;
...

This behavior greatly increases the cost of executing SQL and causes CDC to have high latency in some scenarios.

Describe the feature you'd like

Enables TiCDC to aggregate bulk delete or update operations into a small number of statements, minimizing the cost of executing SQL.

Describe alternatives you've considered

No response

Teachability, Documentation, Adoption, Migration Strategy

No response

@nongfushanquan
Copy link
Contributor

/label affects-6.1

@nongfushanquan
Copy link
Contributor

/area ticdc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-6.1 area/ticdc Issues or PRs related to TiCDC. type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Development

No branches or pull requests

3 participants