Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

schedule: actively push operator #1536

Merged
merged 11 commits into from
May 19, 2019
Merged

schedule: actively push operator #1536

merged 11 commits into from
May 19, 2019

Conversation

nolouch
Copy link
Contributor

@nolouch nolouch commented May 15, 2019 •
edited
Loading

What problem does this PR solve?

For tikv/tikv#4591 and speed up scheduling, we need actively push the operator.

What is changed and how it works?

use a heap to get the needed push operator.

Check List

Tests

@nolouch nolouch added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 15, 2019
@nolouch
remove limit
b8998b1 
Signed-off-by: nolouch <nolouch@gmail.com>
@nolouch nolouch force-pushed the active-operator branch from a0c116c to b8998b1 Compare May 15, 2019 16:58
nolouch added 2 commits May 16, 2019 11:15
@nolouch
add source
dc4ee6a 
Signed-off-by: nolouch <nolouch@gmail.com>
@nolouch
adjust intervel
d49f65f 
Signed-off-by: nolouch <nolouch@gmail.com>
disksing
disksing reviewed May 16, 2019
View reviewed changes
server/coordinator.go Outdated Show resolved Hide resolved
server/coordinator.go Show resolved Hide resolved
server/coordinator.go Outdated Show resolved Hide resolved
server/schedule/operator_controller.go Outdated Show resolved Hide resolved
server/schedule/operator_queue.go Outdated Show resolved Hide resolved
server/schedule/operator_queue.go Outdated Show resolved Hide resolved
@nolouch
address comments
cfd3b57 
Signed-off-by: nolouch <nolouch@gmail.com>
@nolouch nolouch removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 16, 2019
@codecov-io
Copy link

codecov-io commented May 16, 2019 •
edited
Loading

Codecov Report

❗ No coverage uploaded for pull request base (master@6febb24). Click here to learn what that means.
The diff coverage is 91.46%.

Impacted file tree graph

@@            Coverage Diff            @@
##             master    #1536   +/-   ##
=========================================
  Coverage          ?   67.75%           
=========================================
  Files             ?      159           
  Lines             ?    15740           
  Branches          ?        0           
=========================================
  Hits              ?    10664           
  Misses            ?     4112           
  Partials          ?      964
Impacted Files Coverage Δ
tests/cluster.go 84.27% <100%> (ø)
server/schedule/operator_queue.go 100% <100%> (ø)
server/cluster_worker.go 48.33% <100%> (ø)
server/coordinator.go 85.39% <100%> (ø)
server/schedule/operator_controller.go 92.17% <86.53%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6febb24...93d3142. Read the comment docs.

@nolouch nolouch requested review from rleungx and disksing May 16, 2019 08:59
rleungx
rleungx reviewed May 16, 2019
View reviewed changes
server/schedule/operator_controller.go Outdated Show resolved Hide resolved
server/schedule/operator_controller.go Outdated Show resolved Hide resolved
server/schedule/operator_queue.go Outdated Show resolved Hide resolved
server/schedule/operator_controller.go Outdated Show resolved Hide resolved
server/schedule/operator_controller_test.go Outdated Show resolved Hide resolved
@nolouch
address comments
8016601 
Signed-off-by: nolouch <nolouch@gmail.com>
@nolouch nolouch force-pushed the active-operator branch from 7608f63 to 8016601 Compare May 17, 2019 03:12
@nolouch nolouch force-pushed the active-operator branch from 93d3142 to 8016601 Compare May 17, 2019 05:15
shafreeck
shafreeck suggested changes May 17, 2019
View reviewed changes
Copy link
Contributor

@shafreeck shafreeck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the main logic in this PR is implementing a timer with a priority queue, so why not use time.Timer instead?

server/coordinator.go Outdated Show resolved Hide resolved
server/schedule/operator_controller.go Show resolved Hide resolved
server/schedule/operator_queue.go Show resolved Hide resolved
shafreeck
shafreeck approved these changes May 17, 2019
View reviewed changes
Copy link
Contributor

@shafreeck shafreeck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@nolouch
Copy link
Contributor Author

nolouch commented May 17, 2019

/run-all-tests

@nolouch
Copy link
Contributor Author

nolouch commented May 19, 2019

/ok-to-test

@nolouch nolouch merged commit b6150ca into tikv:master May 19, 2019
@nolouch nolouch deleted the active-operator branch May 19, 2019 07:16
nolouch added a commit that referenced this pull request May 23, 2019
@nolouch
*: rebase release 3.0 to master (#1547)
9505c6f 
* tiny clean up code (#1526)

Signed-off-by: Ryan Leung <rleungx@gmail.com>

* config:  enable use region storage by default (#1524)

* config: default enable use region storage

* schedulers: let hot region balance not affect by balance-region-scheduler-limit (#1522)

Signed-off-by: nolouch <nolouch@gmail.com>

* log: do not add stack to error log (#1532)

* *: replace gofail with pingcap/failpoint (#1534)

* *: replace gofail with pingcap/failpoint

Signed-off-by: disksing <i@disksing.com>

* fix typo

Signed-off-by: disksing <i@disksing.com>

* fix typo

Signed-off-by: disksing <i@disksing.com>

* Update server/cluster_test.go

Co-Authored-By: Lonng <chris@lonng.org>

* Update server/tso.go

Co-Authored-By: Lonng <chris@lonng.org>

* Update server/tso.go

Co-Authored-By: Lonng <chris@lonng.org>

* check enable failpoint result

Signed-off-by: disksing <i@disksing.com>

* Update server/cluster.go

Co-Authored-By: Lonng <chris@lonng.org>

* server: set timeout for MoveLeader (#1533)

* server: set timeout for MoveLeader

Signed-off-by: disksing <i@disksing.com>

* client, server: add ScanRegions gRPC protocol support (#1535)

* client, server: support ScanRegions gRPC protocol

Signed-off-by: disksing <i@disksing.com>

* schedule: actively push operator  (#1536)

* schedule: actively push operator

Signed-off-by: nolouch <nolouch@gmail.com>

* *: update some dead links (#1543)

* update links

Signed-off-by: Ryan Leung <rleungx@gmail.com>

* Add windows build script (#1540)

Signed-off-by: Ana Hobden <operator@hoverbear.org>

* fix operator timeout metrics (#1541)

Signed-off-by: Ryan Leung <rleungx@gmail.com>

* *:Rollback config in store when kv.persist failed (#1476)

* tests: independent region sync test (#1545)

* tests: independent syncer region test

Signed-off-by: nolouch <nolouch@gmail.com>

* schedule: operator limit for stores (#1474)

* add store limit for scheduling

Signed-off-by: Ryan Leung <rleungx@gmail.com>

* statistic: add the statistic of flow (#1548)

* store_statistic: add the statistic of flow

Signed-off-by: nolouch <nolouch@gmail.com>
nolouch added a commit that referenced this pull request Aug 16, 2019
@nolouch
schedule: actively push operator (#1536)
559000c 
* schedule: actively push operator

Signed-off-by: nolouch <nolouch@gmail.com>
nolouch added a commit that referenced this pull request Aug 21, 2019
@nolouch
schedule: actively push operator (#1536) (#1686)
1cfe67b 
* schedule: actively push operator  (#1536)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rleungx rleungx rleungx approved these changes

@disksing disksing disksing approved these changes

@shafreeck shafreeck shafreeck approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@nolouch @codecov-io @shafreeck @disksing @rleungx