[QUESTION] How does one setup a global timeout to all requests?

First check

Ticked all checks, then first commitment choice

• I added a very descriptive title to this issue.
• I used the GitHub search to find a similar issue and didn’t find it.
• I searched the FastAPI documentation, with the integrated search.
• I already searched in Google “How to X in FastAPI” and didn’t find any information.
• I already read and followed all the tutorial in the docs and didn’t find an answer.
• I already checked if it is not related to FastAPI but to Pydantic.
• I already checked if it is not related to FastAPI but to Swagger UI.
• I already checked if it is not related to FastAPI but to ReDoc.
• After submitting this, I commit to one of:
  • Read open issues with questions until I find 2 issues where I can help someone and add a comment to help there.
  • I already hit the “watch” button in this repository to receive notifications and I commit to help at least 2 people that ask questions in the future.
  • Implement a Pull Request for a confirmed bug.

Description

Hi there, first of all many thanks for the work on FastAPI - this is now my goto framework for building Python-based REST APIs 😃

My question is about adding a global timeout to any potential request served by the server. My use-case includes occasionally long loading times when I have to load a new model for a given user request, and instead of blocking for 30-50s (which would often timeout on the user side due to default connection timeouts), I would like to return a temporary error whenever any endpoint takes more than a given delay to complete.

Example

Today the only way I found to implement a timeout on every request is to wrap every endpoint method within a context manager like this one:

@contextmanager
def timeout_after(seconds: int):
    # Register a function to raise a TimeoutError on the signal.
    signal.signal(signal.SIGALRM, raise_timeout)
    # Schedule the signal to be sent after `seconds`.
    signal.alarm(seconds)

    try:
        yield
    finally:
        # Unregister the signal so it won't be triggered if the timeout is not reached.
        signal.signal(signal.SIGALRM, signal.SIG_IGN)

def raise_timeout(_, frame):
    raise TimeoutError

# Used as such:
@timeout_after(5)
@app.get("/1/version", tags=["Meta"],
         description="Check if the server is alive, returning the version it runs.",
         response_model=Version,
         response_description="the version of the API currently running.")
async def version() -> Version:
    return current_version

This is however quite cumbersome to add on every single function decorated as an endpoint. Besides, it feels hacky: isn’t there a better way to define app-level timeouts broadly, with a common handler, maybe akin to how ValidationErrors can be managed in a single global handler?

Environment

• OS: [e.g. Linux / Windows / macOS]: Linux
• FastAPI Version [e.g. 0.3.0]: 0.58.0
• Python version: 3.7.7

Additional context

I looked into Starlette’s timeout support to see if that was handled at a lower level. but to no avail.

[MasterScrat]

This is good to return an error message to the user in case of timeout, but is there a way to actually kill the request at the same time so it doesn’t keep using resources?

[PLNech]

@thomas-maschler: thanks for the advice. Unfortunately I’ve tried using Gunicorn’s timeout, but it triggers a full restart of the app, disrupting other users of the service (e.g. by unloading their models from memory). What I’m trying to achieve is rather to enforce a timeout on individual requests, without affecting any other work handled by this worker.