FastAPI is not using enough threads

First Check

• I added a very descriptive title to this issue.
• I used the GitHub search to find a similar issue and didn’t find it.
• I searched the FastAPI documentation, with the integrated search.
• I already searched in Google “How to X in FastAPI” and didn’t find any information.
• I already read and followed all the tutorial in the docs and didn’t find an answer.
• I already checked if it is not related to FastAPI but to Pydantic.
• I already checked if it is not related to FastAPI but to Swagger UI.
• I already checked if it is not related to FastAPI but to ReDoc.

Commit to Help

• I commit to help with one of those options 👆

Example Code

..

Description

When I send too many requests to my fastapi application it spends most of the time doing nothing which I think is because of threading switching and there being not enough threads in the pool. However, the CPU usage is nowhere near 100% and the memory is less than 10% I am wondering how can I force fastapi to use more threads ? I could not find any resource to set the thread to a higher number in the latest version and what is the default number of threads ?

Operating System

Linux

Operating System Details

No response

FastAPI Version

0.73

Python Version

3.9

Additional Context

No response

[Kludex]

Just in case…

import anyio
from fastapi import FastAPI


app = FastAPI()


@app.on_event("startup")
async def startup():
    limiter = anyio.to_thread.current_default_thread_limiter()
    limiter.total_tokens = 100


@app.get("/")
async def home():
    print(anyio.to_thread.current_default_thread_limiter().total_tokens)  # you'll see 100

[Kludex]

You mentioned that you’re using 0.73…

Just do what I said above but instead of changing the value of total_tokens, just print it. You’ll see the default value.