Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

benchmarks is not correct

See original GitHub issue

Hi, i run benchmarks and do not agree with your

Zero:

➜  wrk git:(master) ./wrk -d10s -t8 -c240 http://127.0.0.1:8000/order
Running 10s test @ http://127.0.0.1:8000/order
  8 threads and 240 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   115.38ms   81.09ms 362.51ms   59.20%
    Req/Sec   269.33     65.87   470.00     71.38%
  21495 requests in 10.03s, 5.29MB read
  Socket errors: connect 0, read 103, write 0, timeout 0
Requests/sec:   2143.93
Transfer/sec:    540.17KB
➜  wrk git:(master) ./wrk -d10s -t8 -c240 http://127.0.0.1:8000/order
Running 10s test @ http://127.0.0.1:8000/order
  8 threads and 240 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   107.93ms   36.25ms 179.24ms   75.08%
    Req/Sec   276.98     50.81   424.00     69.00%
  22118 requests in 10.03s, 5.44MB read
  Socket errors: connect 0, read 100, write 0, timeout 0
Requests/sec:   2204.56
Transfer/sec:    555.46KB
➜  wrk git:(master) ./wrk -d10s -t8 -c240 http://127.0.0.1:8000/order
Running 10s test @ http://127.0.0.1:8000/order
  8 threads and 240 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   126.72ms  110.52ms 452.73ms   66.61%
    Req/Sec   267.83     99.47   520.00     65.75%
  21371 requests in 10.03s, 5.26MB read
  Socket errors: connect 0, read 99, write 0, timeout 0
Requests/sec:   2130.75
Transfer/sec:    536.91KB

Sanic:

➜  wrk git:(master) ./wrk -d10s -t8 -c240 http://127.0.0.1:8000/order
Running 10s test @ http://127.0.0.1:8000/order
  8 threads and 240 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   107.09ms  127.92ms   1.35s    95.22%
    Req/Sec   350.38     75.99   524.00     78.76%
  27779 requests in 10.07s, 4.87MB read
  Socket errors: connect 0, read 150, write 0, timeout 0
Requests/sec:   2758.47
Transfer/sec:    495.66KB
➜  wrk git:(master) ./wrk -d10s -t8 -c240 http://127.0.0.1:8000/order
Running 10s test @ http://127.0.0.1:8000/order
  8 threads and 240 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    86.60ms   29.02ms 332.78ms   74.45%
    Req/Sec   347.26     65.74   520.00     74.25%
  27842 requests in 10.08s, 4.89MB read
  Socket errors: connect 0, read 149, write 0, timeout 0
Requests/sec:   2762.66
Transfer/sec:    496.42KB
➜  wrk git:(master) ./wrk -d10s -t8 -c240 http://127.0.0.1:8000/order
Running 10s test @ http://127.0.0.1:8000/order
  8 threads and 240 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    84.77ms   30.17ms 506.58ms   76.15%
    Req/Sec   355.97     59.02   554.00     77.88%
  28529 requests in 10.07s, 5.01MB read
  Socket errors: connect 0, read 149, write 1, timeout 0
Requests/sec:   2832.60
Transfer/sec:    508.98KB

FastAPI:

➜  wrk git:(master) ./wrk -d10s -t8 -c240 http://127.0.0.1:8000/order
Running 10s test @ http://127.0.0.1:8000/order
  8 threads and 240 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   184.36ms  206.19ms   1.23s    90.13%
    Req/Sec   240.48     74.93   392.00     73.57%
  17496 requests in 10.08s, 3.57MB read
  Socket errors: connect 0, read 101, write 0, timeout 0
Requests/sec:   1736.22
Transfer/sec:    362.84KB

➜  wrk git:(master) ./wrk -d10s -t8 -c240 http://127.0.0.1:8000/order
Running 10s test @ http://127.0.0.1:8000/order
  8 threads and 240 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   107.09ms   59.14ms 563.79ms   79.35%
    Req/Sec   289.56     89.17   520.00     69.00%
  23184 requests in 10.06s, 4.73MB read
  Socket errors: connect 0, read 101, write 0, timeout 0
Requests/sec:   2303.92
Transfer/sec:    481.48KB

➜  wrk git:(master) ./wrk -d10s -t8 -c240 http://127.0.0.1:8000/order
Running 10s test @ http://127.0.0.1:8000/order
  8 threads and 240 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   109.62ms   35.28ms 351.88ms   70.78%
    Req/Sec   272.44     47.40   404.00     73.38%
  21811 requests in 10.05s, 4.45MB read
  Socket errors: connect 0, read 101, write 0, timeout 0
Requests/sec:   2169.43
Transfer/sec:    453.38KB

aiohttp:

➜  wrk git:(master) ./wrk -d10s -t8 -c240 http://127.0.0.1:8000/order
Running 10s test @ http://127.0.0.1:8000/order
  8 threads and 240 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   139.98ms  140.77ms   1.35s    94.33%
    Req/Sec   261.75     58.19   431.00     76.31%
  20519 requests in 10.07s, 4.93MB read
  Socket errors: connect 0, read 101, write 0, timeout 0
Requests/sec:   2038.11
Transfer/sec:    501.57KB
➜  wrk git:(master) ./wrk -d10s -t8 -c240 http://127.0.0.1:8000/order
Running 10s test @ http://127.0.0.1:8000/order
  8 threads and 240 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   117.28ms   88.65ms 939.02ms   95.08%
    Req/Sec   282.75     63.12   424.00     78.28%
  22440 requests in 10.06s, 5.39MB read
  Socket errors: connect 0, read 101, write 0, timeout 0
Requests/sec:   2230.80
Transfer/sec:    549.03KB
➜  wrk git:(master) ./wrk -d10s -t8 -c240 http://127.0.0.1:8000/order
Running 10s test @ http://127.0.0.1:8000/order
  8 threads and 240 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   108.08ms   58.23ms 803.84ms   88.32%
    Req/Sec   287.95     61.60   421.00     74.87%
  22947 requests in 10.05s, 5.51MB read
  Socket errors: connect 0, read 96, write 0, timeout 0
Requests/sec:   2282.98
Transfer/sec:    561.84KB

pip freeze:

aiofiles==0.7.0
aiohttp==3.7.4.post0
aioredis @ git+https://github.com/aio-libs/aioredis@ff5a8fe068ebda837d14c3b3777a6182e610854a
anyio==3.2.1
asgiref==3.4.1
async-timeout==3.0.1
attrs==21.2.0
certifi==2021.5.30
chardet==4.0.0
click==8.0.1
coverage==5.5
fastapi==0.66.0
gunicorn==20.1.0
h11==0.12.0
httpcore==0.13.6
httptools==0.2.0
httpx==0.18.2
idna==3.2
iniconfig==1.1.1
msgpack==1.0.2
multidict==5.1.0
packaging==21.0
pluggy==0.13.1
py==1.10.0
pydantic==1.8.2
PyJWT==2.1.0
pyparsing==2.4.7
pytest==6.2.4
pytest-asyncio==0.15.1
pytest-cov==2.12.1
pyzmq==22.1.0
redis==3.5.3
rfc3986==1.5.0
sanic==21.6.0
sanic-routing==0.7.0
sniffio==1.2.0
starlette==0.14.2
toml==0.10.2
typing-extensions==3.10.0.0
ujson==4.0.2
uvicorn==0.14.0
uvloop==0.15.2
websockets==9.1
yarl==1.6.3

Here is the result on MacBook 2,2 GHz 6-Core Intel Core i7, 16 GB

Issue Analytics

State:
Created 2 years ago
Comments:7 (4 by maintainers)

Top GitHub Comments

1reaction

skar404commented, Jul 12, 2021

@Ananto30, my test in MacBook Pro 2018, 2,2 GHz 6-Core Intel Core i7, 16GB, local brew redis

Zero

> ./wrk -d10s -t8 -c240 http://127.0.0.1:8000/order
Running 10s test @ http://127.0.0.1:8000/order
  8 threads and 240 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    58.97ms   58.78ms 479.93ms   81.53%
    Req/Sec   607.07    540.51     2.27k    86.99%
  47960 requests in 10.02s, 11.80MB read
  Socket errors: connect 0, read 102, write 0, timeout 0
Requests/sec:   4785.87
Transfer/sec:      1.18MB

> ./wrk -d10s -t8 -c240 http://127.0.0.1:8000/order
Running 10s test @ http://127.0.0.1:8000/order
  8 threads and 240 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    44.10ms   20.82ms 104.54ms   72.59%
    Req/Sec   685.58    205.24     1.28k    86.12%
  54612 requests in 10.01s, 13.44MB read
  Socket errors: connect 0, read 100, write 0, timeout 0
Requests/sec:   5454.76
Transfer/sec:      1.34MB

> ./wrk -d10s -t8 -c240 http://127.0.0.1:8000/order
Running 10s test @ http://127.0.0.1:8000/order
  8 threads and 240 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    48.67ms   36.33ms 141.08ms   72.68%
    Req/Sec   682.52    275.30     1.40k    80.25%
  54385 requests in 10.01s, 13.38MB read
  Socket errors: connect 0, read 99, write 0, timeout 0
Requests/sec:   5432.37
Transfer/sec:      1.34MB

sanic

> ./wrk -d10s -t8 -c240 http://127.0.0.1:8000/order
Running 10s test @ http://127.0.0.1:8000/order
  8 threads and 240 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    72.83ms  102.84ms   1.29s    95.78%
    Req/Sec   532.89    111.97   828.00     78.58%
  41994 requests in 10.03s, 7.37MB read
  Socket errors: connect 0, read 155, write 5, timeout 0
Requests/sec:   4186.09
Transfer/sec:    752.19KB

> ./wrk -d10s -t8 -c240 http://127.0.0.1:8000/order
Running 10s test @ http://127.0.0.1:8000/order
  8 threads and 240 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    57.91ms   27.21ms 528.90ms   81.51%
    Req/Sec   530.67     84.22   700.00     80.58%
  42124 requests in 10.03s, 7.39MB read
  Socket errors: connect 0, read 152, write 0, timeout 0
Requests/sec:   4198.46
Transfer/sec:    754.41KB

> ./wrk -d10s -t8 -c240 http://127.0.0.1:8000/order
Running 10s test @ http://127.0.0.1:8000/order
  8 threads and 240 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    58.33ms   23.90ms 394.94ms   76.62%
    Req/Sec   522.71     68.04   696.00     76.52%
  41422 requests in 10.03s, 7.27MB read
  Socket errors: connect 0, read 131, write 0, timeout 0
Requests/sec:   4128.58
Transfer/sec:    741.85KB

aiohttp

> ./wrk -d10s -t8 -c240 http://127.0.0.1:8000/order
Running 10s test @ http://127.0.0.1:8000/order
  8 threads and 240 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   111.69ms  175.82ms   1.68s    93.93%
    Req/Sec   402.31     73.10   555.00     83.72%
  31387 requests in 10.03s, 7.54MB read
  Socket errors: connect 0, read 104, write 0, timeout 0
Requests/sec:   3129.75
Transfer/sec:    770.26KB

> ./wrk -d10s -t8 -c240 http://127.0.0.1:8000/order
Running 10s test @ http://127.0.0.1:8000/order
  8 threads and 240 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    76.83ms   44.03ms 558.53ms   75.35%
    Req/Sec   403.51     72.74   580.00     72.73%
  31907 requests in 10.03s, 7.67MB read
  Socket errors: connect 0, read 99, write 0, timeout 0
Requests/sec:   3180.59
Transfer/sec:    782.75KB

> ./wrk -d10s -t8 -c240 http://127.0.0.1:8000/order
Running 10s test @ http://127.0.0.1:8000/order
  8 threads and 240 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    76.97ms   45.23ms 623.97ms   81.98%
    Req/Sec   410.56     68.18   595.00     71.88%
  32564 requests in 10.03s, 7.83MB read
  Socket errors: connect 0, read 102, write 0, timeout 0
  Non-2xx or 3xx responses: 1
Requests/sec:   3247.89
Transfer/sec:    799.32KB

FastAPI

> ./wrk -d10s -t8 -c240 http://127.0.0.1:8000/order
Running 10s test @ http://127.0.0.1:8000/order
  8 threads and 240 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   166.00ms  185.73ms   1.94s    87.89%
    Req/Sec   236.01    104.55   580.00     70.42%
  18729 requests in 10.07s, 3.82MB read
  Socket errors: connect 0, read 102, write 0, timeout 0
Requests/sec:   1859.16
Transfer/sec:    388.54KB

> ./wrk -d10s -t8 -c240 http://127.0.0.1:8000/order
Running 10s test @ http://127.0.0.1:8000/order
  8 threads and 240 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   127.85ms   75.39ms 456.28ms   66.96%
    Req/Sec   239.49     70.89   404.00     71.75%
  19183 requests in 10.07s, 3.91MB read
  Socket errors: connect 0, read 101, write 0, timeout 0
Requests/sec:   1904.81
Transfer/sec:    398.08KB

> ./wrk -d10s -t8 -c240 http://127.0.0.1:8000/order
Running 10s test @ http://127.0.0.1:8000/order
  8 threads and 240 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   134.35ms   87.76ms 750.11ms   67.57%
    Req/Sec   231.22     79.52   460.00     69.62%
  18536 requests in 10.09s, 3.78MB read
  Socket errors: connect 0, read 101, write 0, timeout 0
Requests/sec:   1837.03
Transfer/sec:    383.91KB

0reactions

Ananto30commented, Jul 12, 2021

Thanks a lot for sharing @skar404 🙏