Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure users are distributed evently across hosts during ramp up #2025

Merged
merged 2 commits into from
Feb 22, 2022

Conversation

cyberw
Copy link
Collaborator

@cyberw cyberw commented Feb 21, 2022

No description provided.

@cyberw
Copy link
Collaborator Author

cyberw commented Feb 21, 2022

@mboutet @DennisKrone What do you guys think?

@mboutet
Copy link
Contributor

mboutet commented Feb 21, 2022

Makes sense.

I think you'll also need to add this new sorting logic to:

locust/locust/dispatch.py

Lines 187 to 212 in d34c680

def add_worker(self, worker_node: "WorkerNode") -> None:
"""
This method is to be called when a new worker connects to the master. When
a new worker is added, the users dispatcher will flag that a rebalance is required
and ensure that the next dispatch iteration will be made to redistribute the users
on the new pool of workers.
:param worker_node: The worker node to add.
"""
self._worker_nodes.append(worker_node)
self._worker_nodes = sorted(self._worker_nodes, key=lambda w: w.id)
self._prepare_rebalance()
def remove_worker(self, worker_node: "WorkerNode") -> None:
"""
This method is similar to the above `add_worker`. When a worker disconnects
(because of e.g. network failure, worker failure, etc.), this method will ensure that the next
dispatch iteration redistributes the users on the remaining workers.
:param worker_node: The worker node to remove.
"""
self._worker_nodes = [w for w in self._worker_nodes if w.id != worker_node.id]
if len(self._worker_nodes) == 0:
# TODO: Test this
return
self._prepare_rebalance()

It would be a good idea to also add a unit test to ensure that calling these methods also calls the new sorting logic.

@cyberw cyberw merged commit 1b9c607 into master Feb 22, 2022
@cyberw cyberw deleted the ensure-even-distribution-across-worker-hosts branch February 22, 2022 11:07
@DennisKrone
Copy link
Collaborator

Looks good!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants