Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

scheduler: support multi priorities api for hot-region-scheduler #3899

Merged
merged 12 commits into from
Jul 27, 2021
Merged

scheduler: support multi priorities api for hot-region-scheduler #3899

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
0cdc90c
support api
Yisaer Jul 22, 2021
9641180
support api
Yisaer Jul 22, 2021
7c3b792
support api
Yisaer Jul 22, 2021
467c36e
address the comment
Yisaer Jul 22, 2021
fcd72d2
address the comment
Yisaer Jul 22, 2021
807a828
address the comment
Yisaer Jul 27, 2021
2d2c299
address the comment
Yisaer Jul 27, 2021
610212a
address the comment
Yisaer Jul 27, 2021
e65b232
address the comment
Yisaer Jul 27, 2021
b3e6bd9
address the comment
Yisaer Jul 27, 2021
a0d9fd5
address the comment
Yisaer Jul 27, 2021
0133b1d
Merge branch 'master' into support_multy_priority_api
ti-chi-bot Jul 27, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 18 additions & 8 deletions server/schedulers/hot_region_config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,12 @@ import (
"github.com/unrolled/render"
)

const (
NonePriority = ""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that it might be disturbing for users to think that None means no priority, meaning that each dimension has the same priority

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think our current implement already has priority, it looks:

read-priorities [key, byte]
write-priorities [btye, key]

am I right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated.

BytePriority = "byte"
KeyPriority = "key"
)

// params about hot region.
func initHotRegionScheduleConfig() *hotRegionSchedulerConfig {
return &hotRegionSchedulerConfig{
Expand All @@ -45,6 +51,8 @@ func initHotRegionScheduleConfig() *hotRegionSchedulerConfig {
MaxPeerNum: 1000,
SrcToleranceRatio: 1.05, // Tolerate 5% difference
DstToleranceRatio: 1.05, // Tolerate 5% difference
ReadPriorities: []string{},
WritePriorities: []string{},
}
}

Expand All @@ -59,14 +67,16 @@ type hotRegionSchedulerConfig struct {

// rank step ratio decide the step when calculate rank
// step = max current * rank step ratio
ByteRateRankStepRatio float64 `json:"byte-rate-rank-step-ratio"`
KeyRateRankStepRatio float64 `json:"key-rate-rank-step-ratio"`
QueryRateRankStepRatio float64 `json:"query-rate-rank-step-ratio"`
CountRankStepRatio float64 `json:"count-rank-step-ratio"`
GreatDecRatio float64 `json:"great-dec-ratio"`
MinorDecRatio float64 `json:"minor-dec-ratio"`
SrcToleranceRatio float64 `json:"src-tolerance-ratio"`
DstToleranceRatio float64 `json:"dst-tolerance-ratio"`
ByteRateRankStepRatio float64 `json:"byte-rate-rank-step-ratio"`
KeyRateRankStepRatio float64 `json:"key-rate-rank-step-ratio"`
QueryRateRankStepRatio float64 `json:"query-rate-rank-step-ratio"`
CountRankStepRatio float64 `json:"count-rank-step-ratio"`
GreatDecRatio float64 `json:"great-dec-ratio"`
MinorDecRatio float64 `json:"minor-dec-ratio"`
SrcToleranceRatio float64 `json:"src-tolerance-ratio"`
DstToleranceRatio float64 `json:"dst-tolerance-ratio"`
ReadPriorities []string `json:"read-priorities"`
WritePriorities []string `json:"write-priorities"`
}

func (conf *hotRegionSchedulerConfig) EncodeConfig() ([]byte, error) {
Expand Down
15 changes: 15 additions & 0 deletions tests/pdctl/scheduler/scheduler_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -269,6 +269,8 @@ func (s *schedulerTestSuite) TestScheduler(c *C) {
"minor-dec-ratio": 0.99,
"src-tolerance-ratio": 1.05,
"dst-tolerance-ratio": 1.05,
"read-priorities": []interface{}{},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps, we can use a default priority.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated.

"write-priorities": []interface{}{},
}
c.Assert(conf, DeepEquals, expected1)
mustExec([]string{"-u", pdAddr, "scheduler", "config", "balance-hot-region-scheduler", "set", "src-tolerance-ratio", "1.02"}, nil)
Expand All @@ -277,6 +279,19 @@ func (s *schedulerTestSuite) TestScheduler(c *C) {
mustExec([]string{"-u", pdAddr, "scheduler", "config", "balance-hot-region-scheduler"}, &conf1)
c.Assert(conf1, DeepEquals, expected1)

mustExec([]string{"-u", pdAddr, "scheduler", "config", "balance-hot-region-scheduler", "set", "read-priorities", "key"}, nil)
expected1["read-priorities"] = []interface{}{"key"}
mustExec([]string{"-u", pdAddr, "scheduler", "config", "balance-hot-region-scheduler"}, &conf1)
c.Assert(conf1, DeepEquals, expected1)
mustExec([]string{"-u", pdAddr, "scheduler", "config", "balance-hot-region-scheduler", "set", "read-priorities", "key,byte"}, nil)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What will happen to it if the string is empty or otherwise unparseable？

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

expected1["read-priorities"] = []interface{}{"key", "byte"}
mustExec([]string{"-u", pdAddr, "scheduler", "config", "balance-hot-region-scheduler"}, &conf1)
c.Assert(conf1, DeepEquals, expected1)
mustExec([]string{"-u", pdAddr, "scheduler", "config", "balance-hot-region-scheduler", "set", "read-priorities", "none"}, nil)
expected1["read-priorities"] = []interface{}{}
mustExec([]string{"-u", pdAddr, "scheduler", "config", "balance-hot-region-scheduler"}, &conf1)
c.Assert(conf1, DeepEquals, expected1)

// test show scheduler with paused and disabled status.
checkSchedulerWithStatusCommand := func(args []string, status string, expected []string) {
if args != nil {
Expand Down
20 changes: 19 additions & 1 deletion tools/pd-ctl/pdctl/command/scheduler.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import (

"github.com/pingcap/errors"
"github.com/spf13/cobra"
"github.com/tikv/pd/server/schedulers"
)

var (
Expand Down Expand Up @@ -527,7 +528,24 @@ func postSchedulerConfigCommandFunc(cmd *cobra.Command, schedulerName string, ar
if err != nil {
val = value
}
input[key] = val
if schedulerName == "balance-hot-region-scheduler" && (key == "read-priorities" || key == "write-priorities") {
priorities := make([]string, 0)
if value == "none" {
} else {
for _, priority := range strings.Split(value, ",") {
if priority != schedulers.BytePriority && priority != schedulers.KeyPriority {
cmd.Println(fmt.Sprintf("priority should be one of %s,%s,%s",
schedulers.BytePriority,
schedulers.KeyPriority,
"none"))
}
priorities = append(priorities, priority)
}
}
input[key] = priorities
} else {
input[key] = val
}
postJSON(cmd, path.Join(schedulerConfigPrefix, schedulerName, "config"), input)
}

Expand Down