scheduler: hotspot: solution-based key-rate-aware balance solver #2141

Luffbee · 2020-02-18T09:39:05Z

What problem does this PR solve?

The second step of #2139 .

What is changed and how it works?

Introduce solution based balance solver.

Check List

Tests

Unit test

codecov-io · 2020-02-21T09:50:27Z

Codecov Report

Merging #2141 into master will decrease coverage by 0.22%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #2141      +/-   ##
==========================================
- Coverage   76.27%   76.05%   -0.23%     
==========================================
  Files         195      195              
  Lines       20577    20577              
==========================================
- Hits        15696    15650      -46     
- Misses       3703     3734      +31     
- Partials     1178     1193      +15

Impacted Files	Coverage Δ
server/id/id.go	`77.27% <ø> (ø)`	⬆️
server/grpc_service.go	`58.36% <ø> (+0.21%)`	⬆️
server/config_manager/config_manager.go	`77.01% <ø> (ø)`	⬆️
server/statistics/region.go	`100% <ø> (ø)`	⬆️
server/core/region_tree.go	`93.27% <ø> (ø)`	⬆️
server/api/member.go	`60.83% <ø> (ø)`	⬆️
server/schedule/placement/rule_manager.go	`86.45% <ø> (ø)`	⬆️
tests/config.go	`95.91% <ø> (ø)`	⬆️
server/schedulers/shuffle_region.go	`84.28% <ø> (ø)`	⬆️
server/statistics/store.go	`70.87% <ø> (ø)`	⬆️
... and 118 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4600833...4ac422a. Read the comment docs.

nolouch · 2020-02-24T02:54:14Z

PTAL @disksing @rleungx

server/schedulers/hot_region.go

rleungx · 2020-02-24T08:10:21Z

server/schedulers/hot_region.go

-		if len(ops) > 0 {
-			return ops
-		}
+	// prefer to balance by peer


After removing the retry loop, will it affect the speed of hot spot scheduling?

I believe it won't. The the statistics data is not changed for each iteration, and new balanceSolver will try almost all possible solutions.

rleungx · 2020-02-24T08:28:43Z

server/schedulers/hot_region.go

+		}
+	} else {
+		keyDecRatio := (dstLd.KeyRate + peer.GetKeyRate()) / (srcLd.KeyRate + 1)
+		keyHot := peer.GetKeyRate() >= 10


how about using a constant?

It won't be used in other place, so I don't think it is necessary.

The constant is easy to maintain and the meaning is much more clear.

Add two constants: minHotByteRate and minHotKeyRate.

rleungx · 2020-02-24T08:37:36Z

server/schedulers/hot_region.go

+	case read:
+		return readLeader
+	}
+	return resourceTypeLen


Will it cause panic?

It will only cause panic when the arguments are invalid. I think this is acceptable.

It's better to panic in this function rather than use it?

server/cluster/coordinator.go

server/schedulers/hot_region.go

nolouch · 2020-02-25T07:13:25Z

server/schedulers/hot_region.go

+		if bs.rwTy == write && bs.opTy == transferLeader {
+			lpCmp = sliceLPCmp(
+				minLPCmp(negLoadCmp(sliceLoadCmp(
+					stLdRankCmp(stLdCount, stepRank(bs.maxSrc.Count, bs.rankStep.Count)),


What the meaning of rankStep? it seems always the same in the rank compare.

It is used to discretization. Map float64 to int64. For all values in the interval [rank0 + i * step, rank0 + (i+1) * step), the rank the i.

Got it. If I understand correctly, the ranking i (stepRank return) can only be 0 or 1. Can this distinguish those stores very well?

No, rank i = int64((value - rank0) / step) can be any int64.

Oh, sorry. I missed the stepRatio, can u add some comments about step ratio?

Comments added.

nolouch

lgtm.

nolouch · 2020-02-26T07:20:43Z

/merge

sre-bot · 2020-02-26T07:23:11Z

/run-all-tests

…v#2141)

Luffbee added 7 commits February 18, 2020 16:51

try more solution

f382b47

function betterThan

67d6f24

add pending influence of count

1edb33a

introduce storeLoad[Pred]Cmp

c28388d

remove unused function

e70852b

use resourceType

2a9e000

new byteRateRank; float count

2085ff6

Luffbee added type/enhancement The issue or PR belongs to an enhancement. component/schedule Scheduling logic. labels Feb 18, 2020

Luffbee added this to the v4.0.0-beta.1 milestone Feb 18, 2020

Luffbee requested review from nolouch, rleungx and disksing February 18, 2020 09:39

Luffbee mentioned this pull request Feb 18, 2020

scheduler: hotspot: multidimensional load balance tracking issue #2139

Closed

2 tasks

Luffbee modified the milestones: v4.0.0-beta.1, v4.0.0-rc Feb 20, 2020

Merge branch 'master' into solution-based

eca28c9

Luffbee added 12 commits February 22, 2020 20:25

add key rate for storeLoad, Influence

291be85

make getHotPeers key-rate-aware

1b91e04

make filterDstStores key-rate-aware

228ba6c

make betterThan key-rate-aware

e15c95c

make prepare and influence key-rate-aware

bb52fe3

add metric; tune parameters

ba561d5

write leader: remove byte rate constraint

cb8f562

progressiveRank

37ca8ac

fix old tests

f49baed

remove unused code in test

517247a

add test for hot cache

bff084c

update TestWithPendingInfluence

acd8281

Luffbee added 2 commits February 23, 2020 00:07

add test for key rate balance

5b6abe4

fix typo

ef25256

Luffbee changed the title ~~scheduler: hotspot: solution based balance solver~~ scheduler: hotspot: solution-based key-rate-aware balance solver Feb 22, 2020

disksing reviewed Feb 24, 2020

View reviewed changes

server/schedulers/hot_region.go Outdated Show resolved Hide resolved

server/schedulers/hot_region.go Outdated Show resolved Hide resolved

server/schedulers/hot_region.go Show resolved Hide resolved

server/schedulers/hot_region.go Outdated Show resolved Hide resolved

rleungx reviewed Feb 24, 2020

View reviewed changes

Luffbee added 5 commits February 24, 2020 17:06

make toResourceType panic with invalid arguments

35ec188

remove unnecessary = nil

a7240ad

add constants minHot{Byte,Key}Rate

73feca7

simplify and update comments for calcProgressiveRank

f7b7be2

add constants *RankStepRatio

bae99b8

nolouch reviewed Feb 25, 2020

View reviewed changes

Luffbee added 2 commits February 25, 2020 16:54

add constants: *DecRatio

48e689e

rename getHotPeers and add comments

b0ca960

disksing approved these changes Feb 26, 2020

View reviewed changes

nolouch reviewed Feb 26, 2020

View reviewed changes

add comments for *RankStepRatio

4600833

nolouch approved these changes Feb 26, 2020

View reviewed changes

sre-bot added the status/can-merge Indicates a PR has been approved by a committer. label Feb 26, 2020

Merge branch 'master' into solution-based

4ac422a

sre-bot merged commit c8589fc into tikv:master Feb 26, 2020

Luffbee deleted the solution-based branch February 26, 2020 07:51

nolouch pushed a commit to nolouch/pd that referenced this pull request Apr 10, 2020

scheduler: hotspot: solution-based key-rate-aware balance solver (tik…

044382f

…v#2141)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

scheduler: hotspot: solution-based key-rate-aware balance solver #2141

scheduler: hotspot: solution-based key-rate-aware balance solver #2141

Luffbee commented Feb 18, 2020

codecov-io commented Feb 21, 2020 •

edited

Loading

nolouch commented Feb 24, 2020

rleungx Feb 24, 2020

Luffbee Feb 24, 2020

rleungx Feb 24, 2020

Luffbee Feb 24, 2020

rleungx Feb 24, 2020

Luffbee Feb 24, 2020

rleungx Feb 24, 2020

Luffbee Feb 24, 2020

rleungx Feb 24, 2020

Luffbee Feb 24, 2020

nolouch Feb 25, 2020

Luffbee Feb 25, 2020

nolouch Feb 26, 2020 •

edited

Loading

Luffbee Feb 26, 2020

nolouch Feb 26, 2020 •

edited

Loading

Luffbee Feb 26, 2020

nolouch left a comment

nolouch commented Feb 26, 2020

sre-bot commented Feb 26, 2020

scheduler: hotspot: solution-based key-rate-aware balance solver #2141

scheduler: hotspot: solution-based key-rate-aware balance solver #2141

Conversation

Luffbee commented Feb 18, 2020

What problem does this PR solve?

What is changed and how it works?

Check List

codecov-io commented Feb 21, 2020 • edited Loading

Codecov Report

nolouch commented Feb 24, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nolouch Feb 26, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nolouch Feb 26, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nolouch left a comment

Choose a reason for hiding this comment

nolouch commented Feb 26, 2020

sre-bot commented Feb 26, 2020

codecov-io commented Feb 21, 2020 •

edited

Loading

nolouch Feb 26, 2020 •

edited

Loading

nolouch Feb 26, 2020 •

edited

Loading