FastAPI: Some requests are failing due to 10s timeout
  • 29-May-2023
Lightrun Team
Author Lightrun Team
FastAPI: Some requests are failing due to 10s timeout

FastAPI: Some requests are failing due to 10s timeout

Lightrun Team
Lightrun Team

Explanation of the problem


The problem under consideration revolves around a timeout issue encountered in a production environment utilizing FastAPI for a model prediction service. Despite a moderate load of approximately 2/3 requests per second, certain requests fail to complete within the designated 10-second timeout. Initial investigations focused on isolating potential bottlenecks within the FastAPI framework and related components, including Pydantic, Swagger UI, and ReDoc. Analysis revealed that a significant portion of execution time, approximately 6 seconds, is spent in the starlette.exceptions:ExceptionMiddleware.__call__ segment.

To gain a comprehensive understanding of the problem, a thorough search was conducted in the FastAPI documentation, both using the built-in search feature and external search engines such as Google. The goal was to find relevant information and solutions pertaining to the encountered timeout issue. Additionally, existing discussions and issues related to FastAPI were carefully examined, but no specific resolution addressing this particular problem was found.

To facilitate the troubleshooting process, the gunicorn configuration used in the setup is provided. This configuration specifies parameters such as thread count, worker count, timeout duration, keepalive interval, graceful timeout, and binding port. By sharing this information, the intention is to provide contextual details and enable a thorough analysis of the environment, potentially uncovering any configuration-related factors contributing to the timeout issue.

Operating System: macOS FastAPI Version: 0.65.2 Python Version: 3.7.10

Guidance and support are sought to optimize the FastAPI application, ensuring consistent latency across all requests and eliminating any outliers that result in errors due to the imposed 10-second timeout constraint. Any insights, recommendations, or assistance offered to resolve this issue and enhance the overall performance of the FastAPI application would be greatly appreciated.


Troubleshooting with the Lightrun Developer Observability Platform


Getting a sense of what’s actually happening inside a live application is a frustrating experience, one that relies mostly on querying and observing whatever logs were written during development.
Lightrun is a Developer Observability Platform, allowing developers to add telemetry to live applications in real-time, on-demand, and right from the IDE.

  • Instantly add logs to, set metrics in, and take snapshots of live applications
  • Insights delivered straight to your IDE or CLI
  • Works where you do: dev, QA, staging, CI/CD, and production

Start for free today

Problem solution for: FastAPI: Some requests are failing due to 10s timeout

To address the timeout issue in the FastAPI application and achieve consistent latency for all requests, several steps can be taken:

  1. Identify the root cause of the timeout:
    • Since a significant portion of the execution time is spent in the starlette.exceptions:ExceptionMiddleware.__call__ segment, further investigation is required to understand the underlying cause. This may involve analyzing the middleware configuration, error handling mechanisms, and any custom exception handlers implemented within the application.
  2. Optimize the application code:
    • Evaluate the codebase and identify any performance bottlenecks or inefficient operations that could contribute to the timeout issue. Consider optimizing database queries, reducing unnecessary computations, and improving algorithmic efficiency.
    • Utilize asynchronous programming techniques, such as FastAPI’s support for asynchronous request handlers with async and await, to enhance concurrency and responsiveness of the application.
    • Profile the application to pinpoint specific areas of code that consume excessive time and resources, using tools like cProfile or integrated profiling frameworks.
  3. Review and fine-tune server configuration:
    • Examine the gunicorn configuration provided earlier and ensure it is appropriately tuned for the application’s requirements. Adjust parameters such as worker count, thread count, timeout duration, and keepalive settings to optimize resource allocation and improve performance.
    • Consider using load balancing mechanisms, such as NGINX, in conjunction with FastAPI to distribute incoming requests efficiently and mitigate any potential performance limitations.

Additionally, it’s recommended to leverage FastAPI’s extensive documentation, community forums, and issue trackers to seek guidance, insights, and potential solutions from experienced developers and contributors. Collaborating with the FastAPI community can provide valuable assistance in diagnosing and resolving the timeout issue specific to the application’s setup and workload.


Problems with fastapi


Problem 1: CORS (Cross-Origin Resource Sharing) Issues

One common problem encountered with FastAPI is dealing with CORS issues. When making cross-origin requests from a web browser, the browser enforces security policies that restrict access to resources on different domains. FastAPI applications hosted on one domain may encounter CORS errors when attempting to make requests to another domain.

To solve this issue, FastAPI provides a middleware called fastapi.middleware.cors. By adding this middleware to your application, you can configure the necessary CORS headers to allow cross-origin requests. Here’s an example of how to use the CORS middleware in FastAPI:


from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware

app = FastAPI()

# Configure CORS settings
origins = [



In the code snippet above, we create a list of allowed origins that are allowed to make cross-origin requests. We then add the CORSMiddleware to the FastAPI application using app.add_middleware(), passing in the necessary configuration options. This ensures that the necessary CORS headers are included in the server responses, allowing cross-origin requests to be made successfully.

Problem 2: Request Validation and Serialization

Another common problem in FastAPI is request validation and serialization. FastAPI provides powerful request validation capabilities out of the box, allowing you to define request models and automatically validate and deserialize incoming requests. However, when dealing with complex data structures or nested models, ensuring proper validation and serialization can become challenging.

To address this issue, you can leverage Pydantic, which is integrated with FastAPI and provides robust data validation and serialization features. By creating Pydantic models for your request bodies, you can define the expected structure, types, and constraints. Here’s an example:


from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()

class Item(BaseModel):
    name: str
    price: float"/items/")
async def create_item(item: Item):
    # Access validated and deserialized item properties
    item_name =
    item_price = item.price
    # ...


In the code snippet above, we define a Item Pydantic model that represents the structure of the request body. By specifying the expected types and constraints, FastAPI will automatically validate and deserialize the incoming request. This ensures that only valid data is processed in your API endpoints, reducing the risk of errors and improving the reliability of your application.

Problem 3: Authentication and Authorization

Implementing authentication and authorization mechanisms is a common challenge in web applications, and FastAPI provides various options to address this problem. However, it can still be complex to implement and manage authentication and authorization in a secure and scalable manner.

One solution is to leverage third-party libraries or frameworks for authentication and authorization, such as OAuth 2.0 or JWT (JSON Web Tokens). These libraries provide standardized protocols and mechanisms for authentication and authorization, reducing the complexity of implementation. Here’s an example using OAuth2 and FastAPI OAuth2 support:


from fastapi import FastAPI
from import OAuth2PasswordBearer

app = FastAPI()

oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/token")

async def get_items(token: str = Depends(oauth2_scheme)):
    # Validate and verify the token
    # Perform authorization checks
    # ...


In the code snippet above, we define an OAuth2 password bearer scheme using the OAuth2PasswordBearer class. This enables authentication and authorization through the token parameter in the get_items endpoint. By specifying the Depends(oauth2_scheme) dependency, FastAPI will handle token validation and verification automatically.

By leveraging these authentication and authorization libraries and integrating them with FastAPI, you can ensure that your application’s endpoints are secure and accessible only to authenticated and authorized users.


A brief introduction to fastapi


FastAPI is a modern, fast, and highly efficient web framework for building APIs with Python. It is designed to provide a high-performance alternative to traditional web frameworks by leveraging asynchronous programming and type annotations. Built on top of Starlette and Pydantic, FastAPI combines the best features of these libraries to deliver a seamless development experience.

One of the key advantages of FastAPI is its performance. It utilizes asynchronous programming techniques, such as async and await, to handle multiple concurrent requests efficiently. This allows FastAPI to handle high loads and deliver excellent performance, making it well-suited for applications that require fast response times and scalability. Additionally, FastAPI utilizes type annotations to automatically generate interactive API documentation. With the help of tools like Swagger UI and ReDoc, developers can easily explore and interact with the API endpoints, making it easier to understand and test the API.

FastAPI also provides strong validation and serialization capabilities through its integration with Pydantic. By defining models using Pydantic’s expressive syntax, developers can specify the expected structure, types, and constraints of the request and response data. FastAPI leverages these models to automatically validate and deserialize incoming requests, reducing the likelihood of data-related errors. Furthermore, FastAPI provides built-in support for various authentication and authorization mechanisms, enabling developers to secure their APIs with ease. Overall, FastAPI’s combination of performance, documentation generation, validation, and security features make it a powerful choice for building high-quality web APIs.


Most popular use cases for fastapi


  1. Building High-Performance APIs: FastAPI excels at building high-performance APIs due to its asynchronous nature and efficient request handling. With the help of async and await keywords, developers can write non-blocking code that can handle multiple concurrent requests. Here’s an example of defining an asynchronous route handler in FastAPI:


from fastapi import FastAPI

app = FastAPI()

async def read_item(item_id: int):
    # Perform asynchronous operations here
    # ...
    return {"item_id": item_id}


  1. Interactive API Documentation: FastAPI provides automatic interactive API documentation generation, making it easier for developers and users to explore and understand the available endpoints, request/response models, and input validation rules. FastAPI leverages the type annotations in Python to generate the documentation, which can be accessed through the Swagger UI or ReDoc. Here’s an example of defining a route with request and response models:


from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()

class Item(BaseModel):
    name: str
    price: float"/items/")
async def create_item(item: Item):
    # Process the item data here
    # ...
    return {"message": "Item created"}


  1. Integration with Python Ecosystem: FastAPI seamlessly integrates with various Python libraries and frameworks, allowing developers to leverage the existing ecosystem. It works well with ORMs like SQLAlchemy and databases such as PostgreSQL and MongoDB. FastAPI also supports authentication and authorization mechanisms, including OAuth2 and JWT, making it easy to secure API endpoints. Additionally, FastAPI supports dependency injection, allowing for clean and modular code organization. This integration with the Python ecosystem enables developers to build robust and scalable applications.

It’s Really not that Complicated.

You can actually understand what’s going on inside your live applications. It’s a registration form away.

Get Lightrun

Lets Talk!

Looking for more information about Lightrun and debugging?
We’d love to hear from you!
Drop us a line and we’ll get back to you shortly.

By submitting this form, I agree to Lightrun’s Privacy Policy and Terms of Use.