Why the "RPS" generated by locust is much fewer than other performance testing tools ?

I did load testing on a HTTP interface with several performance testing tools, and I found the ‘RPS’ generated by locust is much fewer than others.

ApacheBench

command: ab -n 1000 -c 80 http://testurl:8000/echo/hello

Benchmark:

Requests per second:    291.38 [#/sec] (mean)
...
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       82  125  87.3    116    2070
Processing:    83  149 250.4    116    2830
Waiting:       82  145 245.0    115    2830
Total:        170  274 278.0    232    4899

Jmeter

set Number of Threads to 80, set Loop Count to 100, and got the value of Throughtout was 270/sec.

Locust

set min_wait = 0 max_wait = 0 in script file, and run locustfile with command: locust -f api.py --no-web -c 80 -r 80 -n 10000 --only-summary

Benchmark:

 Name                                                          # reqs      # fails     Avg     Min     Max  |  Median   req/s
--------------------------------------------------------------------------------------------------------------------------------------------
 GET /echo/hello                                                10000     0(0.00%)    1020     305    3285  |    1000   70.50
--------------------------------------------------------------------------------------------------------------------------------------------
 Total                                                          10000     0(0.00%)                                      70.50

Percentage of the requests completed within given times
 Name                                                           # reqs    50%    66%    75%    80%    90%    95%    98%    99%   100%
--------------------------------------------------------------------------------------------------------------------------------------------
 GET /echo/hello                                                 10000   1000   1100   1100   1100   1200   1400   1700   2100   3285
--------------------------------------------------------------------------------------------------------------------------------------------

[Jasnoor1]

Hi @heyman

While running load at 200 RPS, i am getting some drop on the RPS. Can you please let me know how can we resolve this through locust? Please find the attached reference:

Please let me know how can we pass a constant RPS.

Locust version: 0.13.3

[max-rocket-internet]

There was is an old article from k6 that showed locust being very slow but they configured it with min_wait = 5000 max_wait = 15000, I assume because this was used in an example in the locust docs. They have an updated article now which is very comprehensive: https://k6.io/blog/comparing-best-open-source-load-testing-tools

For comparison, we have run tests up to 30000 RPS (not using FastHttpLocust):

• locust master = 2 cores, 1GB or memory
• locust slaves = 50 slaves at 1 core and 2GB each

Python/locust will likely never match the raw speed of other tools like k6 or the basic tools like wrk/ab etc but ease of the Python language and MUCH easier horizontal scaling on k8s with the master/slave model more than makes up for it IMO.