FastAPI endpoint that contains IO bound and CPU bound tasks. better with or without async??

First Check

• I added a very descriptive title to this issue.
• I used the GitHub search to find a similar issue and didn’t find it.
• I searched the FastAPI documentation, with the integrated search.
• I already searched in Google “How to X in FastAPI” and didn’t find any information.
• I already read and followed all the tutorial in the docs and didn’t find an answer.
• I already checked if it is not related to FastAPI but to Pydantic.
• I already checked if it is not related to FastAPI but to Swagger UI.
• I already checked if it is not related to FastAPI but to ReDoc.

Commit to Help

• I commit to help with one of those options 👆

Example Code

I don't really have code to share

Description

Hello,

I’m trying to move from Flask to FastAPI because it seems better prepared to work concurrently. But the API that I’m trying to build have several endpoints with the same pattern. When they get a request, they connect to a DynamoDB and do a query. After getting that information they do some CPU intensive task (a few seconds) and then it saves the new data in the DB and returns that data. I’m not sure what the best way is in that case. Could the async def syntax help me in this case? What is the current best approach to handle this issue? Thanks,

Operating System

Linux

Operating System Details

No response

FastAPI Version

0.68.0

Python Version

3.8.8

Additional Context

I also did a benchmarking test to compare the two options to define the endpoint. I used JMeter to send 150 requests in 5 seconds to the endpoint. First I launched the API service with the async def syntax and then without the async. The test with the async def syntax was 10 seconds faster to complete. I also noticed that the CPU load during the test was 50% for the async def syntax and 100% for the one without async.

[frankie567]

Here are some technical details about FastAPI and how it’s handling async/sync operations (see: https://fastapi.tiangolo.com/async/?h=technical#very-technical-details):

• If you define an endpoint with async, it’ll run in the event loop; even if there are blocking calls inside (like a blocking call to your database). It means that it can block the server thread and prevent it to answer other requests meanwhile. That’s why it’s preferable to use async-compatible libraries for I/O operations.
• If you define an endpoint without async, as a standard function, FastAPI will automatically send it in a separate thread to avoid the blocking of the event loop. In a sense, it “mimics” an asynchronous behavior.
  • Having to send the task to a separate thread implies a small overhead; so unless you are really doing heavy blocking I/O, it’s not worth it.

Concerning your case, here are my advices:

• Use an async-compatible client for DynamoDB (https://aioboto3.readthedocs.io/en/latest/index.html). This way, you can define every endpoints as async and benefit from async I/O goodness.
• You say that CPU operations take a few seconds. Async I/O won’t help you in this context. If you can live with API endpoints that take a few seconds to answer, well, that should do it.
  • If you really need your API to answer faster, you should consider use a worker process. Basically, it’s a separate process which role is to handle slow background operations. Your API will just send messages in a task queue and respond immediately.
  • On the other side, the worker will handle those tasks as they come. The only pitfall is that your API can’t return the result synchronously. You just know that the operation will be handled at some point. Thus, you’ll have to implement separate endpoints to check for the completion and results of those tasks.
  • Libraries like Celery are designed to do this.

[raphaelauv]

@oriolcmp look this -> https://github.com/encode/starlette/issues/1094

You can very probably close the issue , thank you