Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix hot peer cache threshold unstable when interval unstable #3417

Merged
merged 5 commits into from
Mar 2, 2021

Conversation

lhy1024
Copy link
Contributor

@lhy1024 lhy1024 commented Feb 5, 2021

Signed-off-by: lhy1024 admin@liudos.us

What problem does this PR solve?

Before PR, we add peer to the hot peer cache when its interval is less than normal, which causes topn will increase a lot when the heartbeat interval is unstable. We calculate the threshold by the 60th item in the hot peer cache, so the threshold will reduce to the default threshold while a lot of peers are added to the hot peer cache.

What is changed and how it works?

we add peer to the hot peer cache when its interval is less than normal only when it is hot too.

Check List

Tests

  • Unit test

Release note

Signed-off-by: lhy1024 <admin@liudos.us>
@lhy1024 lhy1024 added the component/scheduler Scheduler logic. label Feb 5, 2021
@ti-chi-bot ti-chi-bot requested review from JmPotato and Yisaer February 5, 2021 06:20
Copy link
Member

@rleungx rleungx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but the test is failed

Copy link
Contributor

@nolouch nolouch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, plz fix the test.

Signed-off-by: lhy1024 <admin@liudos.us>
Signed-off-by: lhy1024 <admin@liudos.us>
@codecov
Copy link

codecov bot commented Feb 22, 2021

Codecov Report

Merging #3417 (70e218d) into master (0d801cd) will decrease coverage by 0.06%.
The diff coverage is 88.88%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3417      +/-   ##
==========================================
- Coverage   74.99%   74.93%   -0.07%     
==========================================
  Files         244      244              
  Lines       23554    23556       +2     
==========================================
- Hits        17664    17651      -13     
- Misses       4314     4322       +8     
- Partials     1576     1583       +7     
Flag Coverage Δ
unittests 74.93% <88.88%> (-0.07%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
server/statistics/hot_peer_cache.go 93.05% <80.00%> (ø)
server/statistics/hot_peer.go 100.00% <100.00%> (ø)
server/schedulers/shuffle_hot_region.go 54.73% <0.00%> (-9.48%) ⬇️
server/schedulers/random_merge.go 60.00% <0.00%> (-3.34%) ⬇️
server/tso/tso.go 69.93% <0.00%> (-3.07%) ⬇️
client/base_client.go 83.23% <0.00%> (-2.32%) ⬇️
server/member/member.go 68.47% <0.00%> (-1.64%) ⬇️
server/core/storage.go 67.68% <0.00%> (-1.53%) ⬇️
server/election/leadership.go 85.88% <0.00%> (-1.18%) ⬇️
server/server.go 72.29% <0.00%> (-0.93%) ⬇️
... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0d801cd...2654165. Read the comment docs.

Copy link
Contributor

@nolouch nolouch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Feb 26, 2021
@nolouch
Copy link
Contributor

nolouch commented Mar 1, 2021

PTAL @rleungx @HunDunDM

Comment on lines +388 to -392
isHot := bytes/interval.Seconds() >= newItem.thresholds[byteDim] || keys/interval.Seconds() >= newItem.thresholds[keyDim]
if !isHot {
return nil
}
if interval.Seconds() >= RegionHeartBeatReportInterval {
isHot := bytes/interval.Seconds() >= newItem.thresholds[byteDim] || keys/interval.Seconds() >= newItem.thresholds[keyDim]
if !isHot {
return nil
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering what's difference here? It seems the result won't be changed with the given isHot and interval.Seconds() >= RegionHeartBeatReportInterval

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it will directly return nil when the interval is less than RegionHeartBeatReportInterval

@ti-chi-bot
Copy link
Member

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • nolouch
  • rleungx

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by writing /lgtm in a comment.
Reviewer can cancel approval by writing /lgtm cancel in a comment.

@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Mar 2, 2021
@nolouch
Copy link
Contributor

nolouch commented Mar 2, 2021

/merge

@ti-chi-bot
Copy link
Member

@nolouch: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 9bfd29d

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Mar 2, 2021
@ti-chi-bot
Copy link
Member

@lhy1024: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot ti-chi-bot merged commit 0e15869 into tikv:master Mar 2, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/scheduler Scheduler logic. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants