add async flush checkpoint feature

[lance6716]

Description

Is your feature request related to a problem? Please describe:

Checkpoint is used when sync binlog to downstream. There're many cases we could advance checkpoint and among them, a case is upstream sending a binlog heartbeat event.

For heartbeat event, currently we wait execution of all DML before it (DM concurrently sync DML), then save the checkpoint. But that will cause the concurrency drop at waiting time.

Describe the feature you'd like:

For heartbeat event, instead of waiting all working goroutine to clear its DML queue, we could find a way like #605, to smoothly update checkpoint. Note that a checkpoint means all binlog event before it is synced. DM tolerates some event after checkpoint has been synced to downstream by "safe mode".

Score

1800

SIG slack channel

sig-migrate

Mentor

@lance6716

Recommended Skills

How DM sync data

[AndreMouche]

/ping

[ti-challenge-bot]

pong! I am challenge bot.

[hidehalo]

/ping

[hidehalo]

/pick-up

[lance6716]

/pick-up

we're refactoring the ti-challenge-bot, I'll manually add the label for you