enhance: Optimize workload based replica selection policy #36181

weiliu1031 · 2024-09-11T08:00:50Z

This PR introduce two new param: toleranceFactor and checkRequestNum, after every checkRequestNum request has been assigned, try to compute querynode's workload score.

if the diff is less than the toleranceFactor, replica selection policy will fallback to round_robin, which reduce the average cost to about 500ns.

if the diff is larger than the toleranceFactor, replica selection policy will compute querynode's score to select the target node with smallest score in every assigment.

mergify · 2024-09-11T08:33:05Z

@weiliu1031 go-sdk check failed, comment rerun go-sdk can trigger the job again.

mergify · 2024-09-11T09:26:25Z

@weiliu1031 E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify · 2024-09-11T13:14:57Z

@weiliu1031 go-sdk check failed, comment rerun go-sdk can trigger the job again.

mergify · 2024-09-11T14:05:11Z

@weiliu1031 E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify · 2024-09-12T02:39:00Z

@weiliu1031 go-sdk check failed, comment rerun go-sdk can trigger the job again.

mergify · 2024-09-12T03:33:01Z

@weiliu1031 E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

weiliu1031 · 2024-09-12T10:05:35Z

before this pr:

goos: linux
goarch: amd64
pkg: github.com/milvus-io/milvus/internal/proxy
cpu: Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz
BenchmarkSelectNode_QNWithSameWorkload-8        	   10000	    115017 ns/op	   14542 B/op	     284 allocs/op
BenchmarkSelectNode_QNWithDifferentWorkload-8   	   10000	    107768 ns/op	   14536 B/op	     284 allocs/op

after this pr:

BenchmarkSelectNode_QNWithSameWorkload
BenchmarkSelectNode_QNWithSameWorkload-8        	 2865908	       493.0 ns/op	      40 B/op	       2 allocs/op
BenchmarkSelectNode_QNWithDifferentWorkload
BenchmarkSelectNode_QNWithDifferentWorkload-8   	 1000000	      1196 ns/op	      40 B/op	       2 allocs/op

mergify · 2024-09-12T11:19:01Z

@weiliu1031 E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

weiliu1031 · 2024-09-13T03:01:33Z

rerun ut

weiliu1031 · 2024-09-13T03:01:44Z

/run-cpu-e2e

codecov · 2024-09-13T05:07:42Z

Codecov Report

Attention: Patch coverage is 95.83333% with 4 lines in your changes missing coverage. Please review.

Project coverage is 82.37%. Comparing base (f652612) to head (2694d09).
Report is 4 commits behind head on master.

Files with missing lines	Patch %	Lines
internal/proxy/look_aside_balancer.go	95.18%	3 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           master   #36181       +/-   ##
===========================================
+ Coverage   71.72%   82.37%   +10.64%     
===========================================
  Files        1276     1276               
  Lines      150629   150653       +24     
===========================================
+ Hits       108039   124100    +16061     
+ Misses      37614    21583    -16031     
+ Partials     4976     4970        -6

Files with missing lines	Coverage Δ
pkg/util/paramtable/component_param.go	`98.29% <100.00%> (+<0.01%)`	⬆️
internal/proxy/look_aside_balancer.go	`96.22% <95.18%> (-3.78%)`	⬇️

... and 286 files with indirect coverage changes

ShineYellow · 2024-09-13T08:59:00Z

rerun go-sdk

yellow-shine · 2024-09-13T09:02:11Z

rerun go-sdk

XuanYang-cn · 2024-09-19T03:18:33Z

pkg/util/paramtable/component_param.go

+		Key:          "proxy.workloadToleranceFactor",
+		Version:      "2.4.12",
+		DefaultValue: "0.1",
+		Doc:          "tolerance factor for query node workload difference",


Please explain more about what's ToleranceFactor

mergify · 2024-09-19T06:33:09Z

@weiliu1031 go-sdk check failed, comment rerun go-sdk can trigger the job again.

mergify · 2024-09-19T06:33:42Z

@weiliu1031 E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

This PR introduce two new param: toleranceFactor and checkRequestNum, after every checkRequestNum request has been assigned, try to compute querynode's workload score. if the diff is less than the toleranceFactor, replica selection policy will fallback to round_robin, which reduce the average cost to about 200ns. if the diff is larger than the toleranceFactor, replica selection policy will compute querynode's score to select the target node with smallest score in every assigment. Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>

XuanYang-cn

/lgtm

congqixia · 2024-09-20T04:32:18Z

/approve

sre-ci-robot · 2024-09-20T04:32:27Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: congqixia, weiliu1031, XuanYang-cn

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~internal/proxy/OWNERS~~ [congqixia]
~~pkg/util/OWNERS~~ [congqixia]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

…36181) issue: milvus-io#35859 This PR introduce two new param: toleranceFactor and checkRequestNum, after every checkRequestNum request has been assigned, try to compute querynode's workload score. if the diff is less than the toleranceFactor, replica selection policy will fallback to round_robin, which reduce the average cost to about 500ns. if the diff is larger than the toleranceFactor, replica selection policy will compute querynode's score to select the target node with smallest score in every assigment. --------- Signed-off-by: Wei Liu <wei.liu@zilliz.com>

…36384) issue: #35859 pr: #36181 This PR introduce two new param: toleranceFactor and checkRequestNum, after every checkRequestNum request has been assigned, try to compute querynode's workload score. if the diff is less than the toleranceFactor, replica selection policy will fallback to round_robin, which reduce the average cost to about 500ns. if the diff is larger than the toleranceFactor, replica selection policy will compute querynode's score to select the target node with smallest score in every assigment. --------- Signed-off-by: Wei Liu <wei.liu@zilliz.com>

sre-ci-robot requested review from bigsheeper and czs007 September 11, 2024 08:00

sre-ci-robot added the size/L Denotes a PR that changes 100-499 lines. label Sep 11, 2024

mergify bot added dco-passed DCO check passed. kind/enhancement Issues or changes related to enhancement labels Sep 11, 2024

weiliu1031 force-pushed the optimize_replica_select branch from d210076 to d8db3e1 Compare September 11, 2024 08:01

weiliu1031 force-pushed the optimize_replica_select branch from d8db3e1 to bb96b15 Compare September 11, 2024 12:52

weiliu1031 force-pushed the optimize_replica_select branch from bb96b15 to d21a606 Compare September 12, 2024 02:15

weiliu1031 force-pushed the optimize_replica_select branch 2 times, most recently from f4f31df to 4206540 Compare September 12, 2024 09:58

weiliu1031 force-pushed the optimize_replica_select branch from 4206540 to 36cebca Compare September 12, 2024 10:06

mergify bot added the ci-passed label Sep 13, 2024

XuanYang-cn reviewed Sep 19, 2024

View reviewed changes

mergify bot removed the ci-passed label Sep 19, 2024

fix review comment

2694d09

Signed-off-by: Wei Liu <wei.liu@zilliz.com>

weiliu1031 force-pushed the optimize_replica_select branch from cba3950 to 2694d09 Compare September 19, 2024 09:24

mergify bot added the ci-passed label Sep 19, 2024

XuanYang-cn approved these changes Sep 20, 2024

View reviewed changes

sre-ci-robot assigned XuanYang-cn Sep 20, 2024

sre-ci-robot added the lgtm label Sep 20, 2024

sre-ci-robot added the approved label Sep 20, 2024

sre-ci-robot merged commit 3b10085 into milvus-io:master Sep 20, 2024
16 checks passed

weiliu1031 mentioned this pull request Sep 20, 2024

enhance: Optimize workload based replica selection policy (#36181) #36384

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

enhance: Optimize workload based replica selection policy #36181

enhance: Optimize workload based replica selection policy #36181

weiliu1031 commented Sep 11, 2024 •

edited

Loading

mergify bot commented Sep 11, 2024

mergify bot commented Sep 11, 2024

mergify bot commented Sep 11, 2024

mergify bot commented Sep 11, 2024

mergify bot commented Sep 12, 2024

mergify bot commented Sep 12, 2024

weiliu1031 commented Sep 12, 2024

mergify bot commented Sep 12, 2024

weiliu1031 commented Sep 13, 2024

weiliu1031 commented Sep 13, 2024

codecov bot commented Sep 13, 2024 •

edited

Loading

ShineYellow commented Sep 13, 2024

yellow-shine commented Sep 13, 2024

XuanYang-cn Sep 19, 2024

mergify bot commented Sep 19, 2024

mergify bot commented Sep 19, 2024

XuanYang-cn left a comment

congqixia commented Sep 20, 2024

sre-ci-robot commented Sep 20, 2024

enhance: Optimize workload based replica selection policy #36181

enhance: Optimize workload based replica selection policy #36181

Conversation

weiliu1031 commented Sep 11, 2024 • edited Loading

mergify bot commented Sep 11, 2024

mergify bot commented Sep 11, 2024

mergify bot commented Sep 11, 2024

mergify bot commented Sep 11, 2024

mergify bot commented Sep 12, 2024

mergify bot commented Sep 12, 2024

weiliu1031 commented Sep 12, 2024

mergify bot commented Sep 12, 2024

weiliu1031 commented Sep 13, 2024

weiliu1031 commented Sep 13, 2024

codecov bot commented Sep 13, 2024 • edited Loading

Codecov Report

ShineYellow commented Sep 13, 2024

yellow-shine commented Sep 13, 2024

XuanYang-cn Sep 19, 2024

Choose a reason for hiding this comment

mergify bot commented Sep 19, 2024

mergify bot commented Sep 19, 2024

XuanYang-cn left a comment

Choose a reason for hiding this comment

congqixia commented Sep 20, 2024

sre-ci-robot commented Sep 20, 2024

weiliu1031 commented Sep 11, 2024 •

edited

Loading

codecov bot commented Sep 13, 2024 •

edited

Loading