Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sink/mq: fix empty value in open protocol #2615

Merged
merged 3 commits into from
Aug 24, 2021
Merged

sink/mq: fix empty value in open protocol #2615

merged 3 commits into from
Aug 24, 2021

Conversation

amyangfei
Copy link
Contributor

What problem does this PR solve?

Fix the open protocol part of #2612

Note we still need to confirm what is the right behavior in protocol canal, canal-json, avro, maxwell for empty row change.

What is changed and how it works?

Ignore empty row change events in open protocol

Check List

Tests

  • Unit test

Release note 

Fix open protocol, don't output an empty value when there is no change in one transaction.

@amyangfei amyangfei added component/open-protocol Open TiCDC protocol component. needs-cherry-pick-release-4.0 Should cherry pick this PR to release-4.0 branch. needs-cherry-pick-release-5.0 Should cherry pick this PR to release-5.0 branch. needs-cherry-pick-release-5.1 Should cherry pick this PR to release-5.1 branch. needs-cherry-pick-release-5.2 Should cherry pick this PR to release-5.2 branch. labels Aug 24, 2021
@ti-chi-bot
Copy link
Member

ti-chi-bot commented Aug 24, 2021 •
edited
Loading

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • hi-rustin
  • liuzix

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added the release-note Denotes a PR that will be considered when it comes time to generate release notes. label Aug 24, 2021
@ti-chi-bot ti-chi-bot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Aug 24, 2021
Rustin170506
Rustin170506 approved these changes Aug 24, 2021
View reviewed changes
Copy link
Member

@Rustin170506 Rustin170506 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

cdc/sink/codec/json.go
// Some transactions could generate empty row change event, such as
// begin; insert into t (id) values (1); delete from t where id=1; commit;
// Just ignore these row changed events
if len(e.Columns) == 0 && len(e.PreColumns) == 0 {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to add the issue link here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO we can trace the issue from commit message and pr description.

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Aug 24, 2021
@Rustin170506
Copy link
Member

/run-all-tests

@amyangfei
Copy link
Contributor Author

/run-all-tests

@amyangfei
Copy link
Contributor Author

/run-integration-tests

@codecov-commenter
Copy link

codecov-commenter commented Aug 24, 2021 •
edited
Loading

Codecov Report

Merging #2615 (b3a7b5d) into master (f3fdd65) will increase coverage by 5.1202%.
The diff coverage is 69.0283%.

@@               Coverage Diff                @@
##             master      #2615        +/-   ##
================================================
+ Coverage   56.0786%   61.1988%   +5.1202%     
================================================
  Files           169        161         -8     
  Lines         20605      17917      -2688     
================================================
- Hits          11555      10965       -590     
+ Misses         7919       5955      -1964     
+ Partials       1131        997       -134

@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Aug 24, 2021
@amyangfei
Copy link
Contributor Author

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 563e5da

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Aug 24, 2021
@nongfushanquan nongfushanquan added the cherry-pick-approved Cherry pick PR approved by release team. label Aug 24, 2021
@amyangfei amyangfei closed this Aug 24, 2021
@amyangfei amyangfei reopened this Aug 24, 2021
@overvenus
Copy link
Member

/merge

@amyangfei
Copy link
Contributor Author

/run-integration-tests

@Rustin170506
Copy link
Member

/run-integration-tests

@ti-chi-bot ti-chi-bot merged commit 13f23cd into pingcap:master Aug 24, 2021
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #2619.

This was referenced Aug 24, 2021
Merged
Merged
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #2620.

@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #2621.

@ti-chi-bot ti-chi-bot mentioned this pull request Aug 24, 2021
Merged
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #2622.

@ti-chi-bot ti-chi-bot mentioned this pull request Aug 24, 2021
Merged
@amyangfei amyangfei deleted the fix-empty-mq-message branch August 24, 2021 15:31
ti-chi-bot added a commit that referenced this pull request Aug 25, 2021
@ti-chi-bot
sink/mq: fix empty value in open protocol (#2615) (#2620)
050413a
ti-chi-bot added a commit that referenced this pull request Aug 25, 2021
@ti-chi-bot
sink/mq: fix empty value in open protocol (#2615) (#2619)
ab1b693
ti-chi-bot added a commit that referenced this pull request Aug 25, 2021
@ti-chi-bot
sink/mq: fix empty value in open protocol (#2615) (#2622)
1d4209c
ti-chi-bot added a commit that referenced this pull request Aug 27, 2021
@ti-chi-bot
sink/mq: fix empty value in open protocol (#2615) (#2621)
ebf7710
@amyangfei amyangfei mentioned this pull request Sep 6, 2021
Closed
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Rustin170506 Rustin170506 Rustin170506 approved these changes

@liuzix liuzix liuzix approved these changes

Assignees
No one assigned
Labels
cherry-pick-approved Cherry pick PR approved by release team. component/open-protocol Open TiCDC protocol component. needs-cherry-pick-release-4.0 Should cherry pick this PR to release-4.0 branch. needs-cherry-pick-release-5.0 Should cherry pick this PR to release-5.0 branch. needs-cherry-pick-release-5.1 Should cherry pick this PR to release-5.1 branch. needs-cherry-pick-release-5.2 Should cherry pick this PR to release-5.2 branch. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@amyangfei @ti-chi-bot @Rustin170506 @codecov-commenter @overvenus @liuzix @nongfushanquan