Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kv/client: don't resolve lock immediately after a region is initialized #2235

Merged
merged 2 commits into from
Jul 12, 2021
Merged

kv/client: don't resolve lock immediately after a region is initialized #2235

merged 2 commits into from
Jul 12, 2021

Conversation

amyangfei
Copy link
Contributor

What problem does this PR solve?

Fix #2188

What is changed and how it works?

Introduce a penalty mechanism

  • If TiCDC keeps receiving unchanged resolved-ts, increase the penalty.
  • If TiCDC receives a new resolved ts, reset the penalty to zero
  • Only resolve ts if the penalty is larger than threshold

Check List

Tests

  • Unit test
  • Integration test

Release note

  • Don't resolve lock immediately after a region is initialized.

@amyangfei amyangfei added status/ptal Could you please take a look? component/kv-client TiKV kv log client component. needs-cherry-pick-release-4.0 Should cherry pick this PR to release-4.0 branch. needs-cherry-pick-release-5.0 Should cherry pick this PR to release-5.0 branch. needs-cherry-pick-release-5.1 Should cherry pick this PR to release-5.1 branch. labels Jul 7, 2021
@ti-chi-bot ti-chi-bot requested review from liuzix and lonng July 7, 2021 07:55
@amyangfei
Copy link
Contributor Author

/run-all-tests

@ti-chi-bot ti-chi-bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Jul 7, 2021
@amyangfei amyangfei requested review from overvenus and removed request for lonng July 7, 2021 07:58
@amyangfei amyangfei force-pushed the fix-resolve-lock-false-positive branch from fe48a78 to 4f3eff7 Compare July 7, 2021 08:00
@amyangfei
Copy link
Contributor Author

/run-all-tests

@amyangfei
Copy link
Contributor Author

/run-kafka-tests

overvenus
overvenus approved these changes Jul 8, 2021
View reviewed changes
cdc/kv/region_worker.go
Comment on lines +314 to +316
rts.ts.resolvedTs = lastResolvedTs
rts.ts.eventTime = time.Now()
rts.ts.penalty = 0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you reset the penalty in func Upsert()?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Upsert here is a insert operation actually, so we must reset penalty outside

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Jul 8, 2021
@ti-chi-bot
Copy link
Member

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • liuzix
  • overvenus

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Jul 12, 2021
@amyangfei
Copy link
Contributor Author

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 4f3eff7

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jul 12, 2021
@amyangfei
Copy link
Contributor Author

/run-all-tests

@amyangfei amyangfei added this to the v5.1.1 milestone Jul 12, 2021
@codecov-commenter
Copy link

Codecov Report

Merging #2235 (838fd10) into master (b6451b6) will increase coverage by 2.0106%.
The diff coverage is 66.3610%.

@@               Coverage Diff                @@
##             master      #2235        +/-   ##
================================================
+ Coverage   52.7179%   54.7285%   +2.0106%     
================================================
  Files           164        171         +7     
  Lines         17679      20852      +3173     
================================================
+ Hits           9320      11412      +2092     
- Misses         7348       8333       +985     
- Partials       1011       1107        +96

@ti-chi-bot ti-chi-bot merged commit bda60ed into pingcap:master Jul 12, 2021
@amyangfei amyangfei deleted the fix-resolve-lock-false-positive branch July 12, 2021 06:17
@ti-chi-bot ti-chi-bot mentioned this pull request Jul 12, 2021
Merged
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #2264.

@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #2265.

This was referenced Jul 12, 2021
Merged
Merged
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #2266.

ti-chi-bot added a commit that referenced this pull request Jul 12, 2021
@ti-chi-bot
kv/client: don't resolve lock immediately after a region is initializ…
8ad0e8d 
…ed (#2235) (#2264)
ti-chi-bot added a commit that referenced this pull request Jul 14, 2021
@ti-chi-bot
kv/client: don't resolve lock immediately after a region is initializ…
4f584d5 
…ed (#2235) (#2266)
ti-chi-bot added a commit that referenced this pull request Jul 17, 2021
@ti-chi-bot
kv/client: don't resolve lock immediately after a region is initializ…
5c29a5d 
…ed (#2235) (#2265)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@overvenus overvenus overvenus approved these changes

@liuzix liuzix liuzix approved these changes

Assignees
No one assigned
Labels
component/kv-client TiKV kv log client component. needs-cherry-pick-release-4.0 Should cherry pick this PR to release-4.0 branch. needs-cherry-pick-release-5.0 Should cherry pick this PR to release-5.0 branch. needs-cherry-pick-release-5.1 Should cherry pick this PR to release-5.1 branch. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. status/ptal Could you please take a look?
Projects
None yet
Milestone
v5.1.1
Development

Successfully merging this pull request may close these issues.

kv client resolves lock immediately after ticdc is up
5 participants
@amyangfei @ti-chi-bot @codecov-commenter @overvenus @liuzix