Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

skip the "cancled" ddl in ddl history #1229

Merged
merged 2 commits into from
May 11, 2023
Merged

skip the "cancled" ddl in ddl history #1229

merged 2 commits into from
May 11, 2023

Conversation

okJiang
Copy link
Member

@okJiang okJiang commented May 5, 2023

What problem does this PR solve?

Issue Number: close #1228

What is changed and how it works?

as title

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Code changes

  • Has exported function/method change
  • Has exported variable/fields change
  • Has interface methods change
  • Has persistent data change

Side effects

  • Possible performance regression
  • Increased code complexity
  • Breaking backward compatibility

Related changes

  • Need to cherry-pick to the release branch
  • Need to update the documentation
  • Need to update the tidb-ansible repository
  • Need to be included in the release note

Release note

  • No release note

@ti-chi-bot
Copy link

ti-chi-bot bot commented May 5, 2023 •
edited
Loading

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • GMHDBJD
  • lichunzhu

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot bot added the size/S label May 5, 2023
@okJiang okJiang requested a review from lichunzhu May 5, 2023 06:25
lichunzhu
lichunzhu reviewed May 5, 2023
View reviewed changes
Copy link
Contributor

@lichunzhu lichunzhu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please help add a unit test in TestNewSyncer function? Thanks a lot!

drainer/syncer.go Outdated
log.Info("skip ddl due to the failed ddl", zap.String("sql", sql), zap.Int64("commit ts", commitTS))
shouldSkip = true
} else if s.cfg.SyncDDL {
schema, table, err = s.schema.getSchemaTableAndDelete(b.job.BinlogInfo.SchemaVersion)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can't skip this step when s.cfg.SyncDDL == false. The version2SchemaTable map will keep increasing and cause an oom issue. What's more, schema, table info will be needed when we do the following steps.

drainer/syncer.go Outdated

if b.job.BinlogInfo.SchemaVersion == 0 {
log.Info("skip ddl due to the failed ddl", zap.String("sql", sql), zap.Int64("commit ts", commitTS))
shouldSkip = true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think just skipping this kind of jobs and then append fake binlogs is okay.

Suggested change
shouldSkip = true
appendFakeBinlogIfNeeded(nil, commitTS)
continue

@ti-chi-bot ti-chi-bot bot added size/M and removed size/S labels May 11, 2023
@okJiang
Copy link
Member Author

okJiang commented May 11, 2023

ptal again ~ @lichunzhu

@okJiang
Copy link
Member Author

okJiang commented May 11, 2023

/merge

@ti-chi-bot
Copy link

ti-chi-bot bot commented May 11, 2023

@okJiang: /merge in this pull request requires 2 approval(s).

In response to this:

/merge

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

GMHDBJD
GMHDBJD approved these changes May 11, 2023
View reviewed changes
Copy link
Contributor

@GMHDBJD GMHDBJD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@okJiang
Copy link
Member Author

okJiang commented May 11, 2023

/merge

@ti-chi-bot
Copy link

ti-chi-bot bot commented May 11, 2023

This pull request has been accepted and is ready to merge.

Commit hash: 32c6f7b

@okJiang
Copy link
Member Author

okJiang commented May 11, 2023

/cherry-pick release-7.1

@ti-chi-bot ti-chi-bot bot merged commit a604070 into pingcap:master May 11, 2023
@ti-chi-bot ti-chi-bot mentioned this pull request May 11, 2023
Merged
@ti-chi-bot
Copy link
Member

@okJiang: new pull request created to branch release-7.1: #1230.

In response to this:

/cherry-pick release-7.1

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot
Copy link
Member

@okJiang: new pull request could not be created: failed to create pull request against pingcap/tidb-binlog#release-7.1 from head ti-chi-bot:cherry-pick-1229-to-release-7.1: status code 422 not one of [201], body: {"message":"Validation Failed","errors":[{"resource":"PullRequest","code":"custom","message":"A pull request already exists for ti-chi-bot:cherry-pick-1229-to-release-7.1."}],"documentation_url":"https://docs.github.com/rest/reference/pulls#create-a-pull-request"}

In response to this:

/cherry-pick release-7.1

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot bot pushed a commit that referenced this pull request May 11, 2023
@ti-chi-bot
skip the "cancled" ddl in ddl history (#1229) (#1230)
fcfb1d0 
close #1228
@okJiang
Copy link
Member Author

okJiang commented May 16, 2023

/cherry-pick release-6.5

@ti-chi-bot ti-chi-bot mentioned this pull request May 16, 2023
Merged
@ti-chi-bot
Copy link
Member

@okJiang: new pull request created to branch release-6.5: #1232.

In response to this:

/cherry-pick release-6.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot bot pushed a commit that referenced this pull request May 23, 2023
@ti-chi-bot
skip the "cancled" ddl in ddl history (#1229) (#1232)
a945b00 
close #1228
@ti-chi-bot ti-chi-bot mentioned this pull request Jun 30, 2023
Merged
ti-chi-bot bot pushed a commit that referenced this pull request Jul 3, 2023
@ti-chi-bot
skip the "cancled" ddl in ddl history (#1229) (#1230) (#1255)
9f5e669 
close #1228
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@GMHDBJD GMHDBJD GMHDBJD approved these changes

@lichunzhu lichunzhu lichunzhu approved these changes

Assignees
No one assigned
Labels
size/M status/can-merge status/LGT2
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

When binlog encounters a failed DDL statement (contains if not exists / if exists), it will report an error
4 participants
@okJiang @ti-chi-bot @lichunzhu @GMHDBJD