Skip to content
This repository has been archived by the owner on Nov 24, 2023. It is now read-only.

syncer: don't skip jobs from same event when comparing table checkpoint #1752

Merged
merged 11 commits into from
Jun 17, 2021

Conversation

lance6716
Copy link
Collaborator

@lance6716 lance6716 commented Jun 5, 2021

What problem does this PR solve?

close #1751

after all_mode paused, we could see

MySQL [all_mode]> select * from t2;
+----+-------+---------------------+
| id | name  | ts                  |
+----+-------+---------------------+
|  1 | Arya  | 2021-06-05 18:01:52 |
|  2 | Bran  | 2021-05-11 10:01:05 |
|  3 | Sansa | 2021-06-05 18:01:52 |
| 10 | 10    | 2021-06-05 18:02:02 |
| 20 | 20    | 2021-06-05 18:02:02 |
| 30 | 30    | 2021-06-05 18:02:02 |
+----+-------+---------------------+
6 rows in set (0.001 sec)

MySQL [all_mode]> select * from t1;
+----+---------+---------------------+---------------------+
| id | name    | dt                  | ts                  |
+----+---------+---------------------+---------------------+
|  1 | ar"ya   | 2021-06-05 18:01:52 | 2021-06-05 18:01:52 |
|  2 | catelyn | 2021-05-11 10:01:05 | 2021-05-11 10:01:05 |
|  3 | lalala  | NULL                | 2021-06-05 18:01:52 |
| 10 | 10      | NULL                | 2021-06-05 18:02:01 |
| 30 | 30      | NULL                | 2021-06-05 18:02:02 |
+----+---------+---------------------+---------------------+
5 rows in set (0.001 sec)

t1 losts a row with id = 20 of https://github.com/pingcap/dm/pull/1752/files#diff-5cc7189b51f6219ea0ce1c9920f92f84cee41b3fee093894a72ca5ff1c615b4aR2

What is changed and how it works?

Check List

Tests

  • Integration test

Code changes

  • Has interface methods change

Related changes

  • Need to cherry-pick to the release branch
  • Need to be included in the release note

@lance6716 lance6716 added this to the v2.0.4 milestone Jun 7, 2021
@lance6716 lance6716 changed the title [WIP] add a test to reproduce BUG syncer: don't skip jobs from same event when comparing table checkpoint Jun 9, 2021
@lance6716 lance6716 added needs-cherry-pick-release-2.0 This PR should be cherry-picked to release-2.0. Remove this label after cherry-picked to release-2.0 needs-update-release-note This PR should be added into release notes. Remove this label once the release notes are updated labels Jun 9, 2021
@lance6716 lance6716 removed this from the v2.0.4 milestone Jun 9, 2021
@GMHDBJD GMHDBJD added this to the v2.0.4 milestone Jun 10, 2021
Copy link
Collaborator

@GMHDBJD GMHDBJD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot added the status/LGT1 One reviewer already commented LGTM label Jun 15, 2021
Copy link
Contributor

@Ehco1996 Ehco1996 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rest LGTM

syncer/checkpoint.go Show resolved Hide resolved
syncer/checkpoint.go Show resolved Hide resolved
syncer/checkpoint.go Show resolved Hide resolved
@ti-chi-bot
Copy link
Member

@Ehco1996: Thanks for your review. The bot only counts approvals from reviewers and higher roles in list, but you're still welcome to leave your comments.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot
Copy link
Member

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • GMHDBJD
  • glorv

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added status/LGT2 Two reviewers already commented LGTM, ready for merge and removed status/LGT1 One reviewer already commented LGTM labels Jun 17, 2021
@glorv
Copy link
Collaborator

glorv commented Jun 17, 2021

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 4c80409

@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #1781.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
needs-cherry-pick-release-2.0 This PR should be cherry-picked to release-2.0. Remove this label after cherry-picked to release-2.0 needs-update-release-note This PR should be added into release notes. Remove this label once the release notes are updated size/L status/can-merge status/LGT2 Two reviewers already commented LGTM, ready for merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

DM may lost data when resume from error in non GTID mode
5 participants