question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Users with `fixed_count` not being relocated after rebalance

See original GitHub issue

Describe the bug

Running Locust with multiple workers, where UserA has fixed_count = 1 and UserB has no weight attributes.

As expected when starting 4 workers and 8 users, 1 UserA is started with 7 UserBs. Though when the worker UserA is running on is terminated, Locust rebalances such that theres now 8 UserBs and no UserAs. The same issue seems sometimes occur when just scaling up the number of workers, triggering a rebalance.

Adding some logs to the code, looks like self._try_dispatch_fixed is False on the rebalance (dispatch.py#L387) - which is only set to True when it first runs or when the number of users is reduced.

I’m happy have a go at fixing this - though adding issue for visibility.

Steps to reproduce

# locustfile.py
class UserA:
  fixed_count = 1
  # ...

class UserB:
  # ...
  1. Add 4 workers
  2. Add 1 user: This adds a single UserA as expected
  3. Add 7 more users: This adds 7 UserBs as expected
  4. Terminate the worker running the single UserA
  5. After the rebalance now have 8 UserBs and no UserAs
  6. Adding 2 more users (taking total to 10) still has no UserAs
All users spawned: {"UserA": 1, "UserB": 0} (1 total users)
All users spawned: {"UserA": 1, "UserB": 7} (8 total users)
All users spawned: {"UserA": 0, "UserB": 10} (10 total users)

Environment

  • OS: python:3.9 container
  • Locust version: running master
  • Locust command line that you ran: /usr/local/bin/python /usr/local/bin/locust --master --locustfile .../locustfile.py
  • Locust file contents (anonymized if necessary)
class UserA:
  fixed_count = 1
  # ...

class UserB:
  # ...

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:7 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
EzR1d3rcommented, May 6, 2022

@cyberw @andydunstall Hi! It is not known issue. I dont remember rebalance in details right now. Does a master request from workers remained users before rebalancing? Overall, changing self._try_dispatch_fixed to True looks reasonable. And I guess, we need a test for this case, may be in locust/test/test_main.py?

0reactions
andydunstallcommented, May 8, 2022

@EzR1d3r Ok great thanks - I’ve added a PR here https://github.com/locustio/locust/pull/2093

Read more comments on GitHub >

github_iconTop Results From Across the Web

Solving My Weird Kafka Rebalancing Problems & Explaining ...
The synchronization barrier was moved in between these two rebalancings. First, partitions are revoked and not assigned. Then, and after the ...
Read more >
S&P DJI's Equity Indices Policies & Practices Methodology
Current index constituents are generally not deleted at a rebalancing solely for the reason of being the target of a pending acquisition.
Read more >
locust/CHANGELOG and locust Releases (Page 8) | LibHunt
Editing a running test in the Web UI with class-picker restarts user count back at 0 # ... Users with fixed_count not being...
Read more >
Kafka 0.9.0 end offset moved back on rebalance
I'm having a problem when there is a rebalance. Offset is moved back as seen in the example log below. INFO [2016-02-25 04:39:03,628]...
Read more >
Cooperative Rebalancing in the Kafka Consumer, Streams ...
Everyone wants their infrastructure to be highly available, ... The others pose a challenge since consumers do not know a priori which of ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found