Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

--limit-max-requests not working

See original GitHub issue

Hi! I’m trying to figure out why my workers are not restarting as expected when using the --limit-max-requests flag. I ran uvicorn in debug mode and noticed that the self.server_state.total_requests count is not increasing (stays at 0 after each request) so self.server_state.total_requests >= self.config.limit_max_requests never returns True. When looking into where the total_requests was used, I noticed that the protocols.http.[auto/h11/httptools] were never getting called. I tried forcing the --http h11 and --http httptools parameters, without any change in behavior. Any help would be appreciated!

Issue Analytics

State:
Created 4 years ago
Comments:5 (2 by maintainers)

Top GitHub Comments

2reactions

jsjeannottecommented, Sep 23, 2019

To reproduce

Content of aiohttp_main.py:

import logging
import os
from aiohttp import web

logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
logging.info(f"WORKER STARTING with pid {os.getpid()}")

routes = web.RouteTableDef()


@routes.get('/')
async def healthcheck(request):
    return web.json_response(dict(status="Ok"))


app = web.Application()
app.add_routes(routes)

Content of fastapi_main.py

import logging
import os
from fastapi import FastAPI

logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
logging.info(f"WORKER STARTING with pid {os.getpid()}")
app = FastAPI()


@app.get("/")
def root():
    return dict(status="Ok")

Here is the output from each experiment showing that with the aiohttp.GunicornWebWorker, the worker is getting properly restarted, but not with the uvicorn.workers.UvicornWorker (generated the load with: for i in $(seq 1 20); do curl http://0.0.0.0:8000; done):

> gunicorn aiohttp_main:app --bind 0.0.0.0:8000 -k aiohttp.GunicornWebWorker --workers 2 --max-requests 5

Output:

[2019-09-23 10:17:36 -0700] [55697] [INFO] Starting gunicorn 19.9.0
[2019-09-23 10:17:36 -0700] [55697] [INFO] Listening at: http://0.0.0.0:8000 (55697)
[2019-09-23 10:17:36 -0700] [55697] [INFO] Using worker: aiohttp.GunicornWebWorker
[2019-09-23 10:17:36 -0700] [55700] [INFO] Booting worker with pid: 55700
2019-09-23 10:17:36,775 - INFO - WORKER STARTING with pid 55700
[2019-09-23 10:17:36 -0700] [55701] [INFO] Booting worker with pid: 55701
2019-09-23 10:17:36,849 - INFO - WORKER STARTING with pid 55701
[2019-09-23 10:17:39 -0700] [55700] [INFO] Max requests, shutting down: <Worker 55700>
[2019-09-23 10:17:39 -0700] [55700] [INFO] Worker exiting (pid: 55700)
[2019-09-23 10:17:39 -0700] [55740] [INFO] Booting worker with pid: 55740
2019-09-23 10:17:39,852 - INFO - WORKER STARTING with pid 55740
[2019-09-23 10:17:39 -0700] [55701] [INFO] Max requests, shutting down: <Worker 55701>
[2019-09-23 10:17:39 -0700] [55701] [INFO] Worker exiting (pid: 55701)
[2019-09-23 10:17:39 -0700] [55741] [INFO] Booting worker with pid: 55741
2019-09-23 10:17:39,944 - INFO - WORKER STARTING with pid 55741

> gunicorn fastapi_main:app --bind 0.0.0.0:8000 -k uvicorn.workers.UvicornWorker --workers 2 --max-requests 5

Output:

[2019-09-23 10:08:07 -0700] [55306] [INFO] Starting gunicorn 19.9.0
[2019-09-23 10:08:07 -0700] [55306] [INFO] Listening at: http://0.0.0.0:8000 (55306)
[2019-09-23 10:08:07 -0700] [55306] [INFO] Using worker: uvicorn.workers.UvicornWorker
[2019-09-23 10:08:07 -0700] [55309] [INFO] Booting worker with pid: 55309
[2019-09-23 10:08:07 -0700] [55310] [INFO] Booting worker with pid: 55310
email-validator not installed, email fields will be treated as str.
To install, run: pip install email-validator
email-validator not installed, email fields will be treated as str.
To install, run: pip install email-validator
2019-09-23 10:08:07,372 - INFO - WORKER STARTING with pid 55309
[2019-09-23 10:08:07 -0700] [55309] [INFO] Started server process [55309]
[2019-09-23 10:08:07 -0700] [55309] [INFO] Waiting for application startup.
2019-09-23 10:08:07,392 - INFO - WORKER STARTING with pid 55310
[2019-09-23 10:08:07 -0700] [55310] [INFO] Started server process [55310]
[2019-09-23 10:08:07 -0700] [55310] [INFO] Waiting for application startup.
[2019-09-23 10:10:50 -0700] [55310] [INFO] ('127.0.0.1', 60134) - "GET / HTTP/1.1" 200
[2019-09-23 10:10:50 -0700] [55310] [INFO] ('127.0.0.1', 60135) - "GET / HTTP/1.1" 200
[2019-09-23 10:10:50 -0700] [55310] [INFO] ('127.0.0.1', 60136) - "GET / HTTP/1.1" 200
[2019-09-23 10:10:50 -0700] [55310] [INFO] ('127.0.0.1', 60137) - "GET / HTTP/1.1" 200
[2019-09-23 10:10:50 -0700] [55310] [INFO] ('127.0.0.1', 60138) - "GET / HTTP/1.1" 200
[2019-09-23 10:10:50 -0700] [55310] [INFO] ('127.0.0.1', 60139) - "GET / HTTP/1.1" 200
[2019-09-23 10:10:50 -0700] [55310] [INFO] ('127.0.0.1', 60140) - "GET / HTTP/1.1" 200
[2019-09-23 10:10:50 -0700] [55310] [INFO] ('127.0.0.1', 60141) - "GET / HTTP/1.1" 200
[2019-09-23 10:10:50 -0700] [55310] [INFO] ('127.0.0.1', 60142) - "GET / HTTP/1.1" 200
[2019-09-23 10:10:50 -0700] [55309] [INFO] ('127.0.0.1', 60143) - "GET / HTTP/1.1" 200
[2019-09-23 10:10:50 -0700] [55310] [INFO] ('127.0.0.1', 60144) - "GET / HTTP/1.1" 200
[2019-09-23 10:10:50 -0700] [55309] [INFO] ('127.0.0.1', 60145) - "GET / HTTP/1.1" 200
[2019-09-23 10:10:50 -0700] [55310] [INFO] ('127.0.0.1', 60146) - "GET / HTTP/1.1" 200
[2019-09-23 10:10:50 -0700] [55310] [INFO] ('127.0.0.1', 60147) - "GET / HTTP/1.1" 200
[2019-09-23 10:10:50 -0700] [55310] [INFO] ('127.0.0.1', 60148) - "GET / HTTP/1.1" 200
[2019-09-23 10:10:50 -0700] [55310] [INFO] ('127.0.0.1', 60149) - "GET / HTTP/1.1" 200
[2019-09-23 10:10:50 -0700] [55310] [INFO] ('127.0.0.1', 60150) - "GET / HTTP/1.1" 200
[2019-09-23 10:10:50 -0700] [55310] [INFO] ('127.0.0.1', 60151) - "GET / HTTP/1.1" 200
[2019-09-23 10:10:50 -0700] [55310] [INFO] ('127.0.0.1', 60152) - "GET / HTTP/1.1" 200
[2019-09-23 10:10:50 -0700] [55309] [INFO] ('127.0.0.1', 60153) - "GET / HTTP/1.1" 200

2reactions

Hanaasagicommented, Sep 19, 2019

--limit-max-requests only terminates the worker process. The manager process will not restart it again. You can use uvicorn with gunicorn.

Gunicorn is a mature, fully featured server and process manager.

Uvicorn includes a Gunicorn worker class allowing you to run ASGI applications, with all of Uvicorn’s performance benefits, while also giving you Gunicorn’s fully-featured process management.

This allows you to increase or decrease the number of worker processes on the fly, restart worker processes gracefully, or perform server upgrades without downtime.

Ref: https://www.uvicorn.org/deployment/