RuntimeError: Stream consumed when reading body with startlette (bug since 1.9.10)

How do you use Sentry?

Sentry Saas (sentry.io)

Version

1.9.10

Steps to Reproduce

Server:

import sentry_sdk

from fastapi import FastAPI, Request

sentry_sdk.init()

app = FastAPI()


@app.post("/")
async def index(request: Request):
    r = await request.body()
    return {"Hello": "World"}

Run with uvicorn:

uvicorn app:app --port 5000

As client do:

#! /usr/bin/python
import httpx

import asyncio


async def main():
    image = b"\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\x01\x00\x00\x00\x01\x08\x06\x00\x00\x00\x1f\x15\xc4\x89\x00\x00\x00\rIDATx\xdac\xfc\xcf\xf0\xbf\x1e\x00\x06\x83\x02\x7f\x94\xad\xd0\xeb\x00\x00\x00\x00IEND\xaeB`\x82"
    files = {"file": ("test.png", image)}
    async with httpx.AsyncClient() as client:
        r = await client.post(
            "http://localhost:5000/",
            files=files,
        )
        print(r)


asyncio.run(main())

Expected Result

To succeed.

Actual Result

ERROR: Exception in ASGI application Traceback (most recent call last): File “/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py”, line 404, in run_asgi result = await app( # type: ignore[func-returns-value] File “/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py”, line 78, in call return await self.app(scope, receive, send) File “/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/fastapi/applications.py”, line 270, in call await super().call(scope, receive, send) File “/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py”, line 293, in _sentry_patched_asgi_app return await middleware(scope, receive, send) File “/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py”, line 138, in _run_asgi3 return await self._run_app(scope, lambda: self.app(scope, receive, send)) File “/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py”, line 187, in _run_app raise exc from None File “/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py”, line 182, in _run_app return await callback() File “/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/starlette/applications.py”, line 124, in call await self.middleware_stack(scope, receive, send) File “/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py”, line 98, in _create_span_call await old_call(*args, **kwargs) File “/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/starlette/middleware/errors.py”, line 184, in call raise exc File “/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/starlette/middleware/errors.py”, line 162, in call await self.app(scope, receive, _send) File “/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py”, line 191, in _sentry_exceptionmiddleware_call await old_call(self, scope, receive, send) File “/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py”, line 98, in _create_span_call await old_call(*args, **kwargs) File “/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/starlette/middleware/exceptions.py”, line 79, in call raise exc File “/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/starlette/middleware/exceptions.py”, line 68, in call await self.app(scope, receive, sender) File “/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py”, line 98, in _create_span_call await old_call(*args, **kwargs) File “/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py”, line 21, in call raise e File “/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py”, line 18, in call await self.app(scope, receive, send) File “/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/starlette/routing.py”, line 706, in call await route.handle(scope, receive, send) File “/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/starlette/routing.py”, line 276, in handle await self.app(scope, receive, send) File “/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/starlette/routing.py”, line 66, in app response = await func(request) File “/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/sentry_sdk/integrations/fastapi.py”, line 106, in _sentry_app return await old_app(*args, **kwargs) File “/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/fastapi/routing.py”, line 231, in app raw_response = await run_endpoint_function( File “/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/fastapi/routing.py”, line 160, in run_endpoint_function return await dependant.call(**values) File “/home/sevaho/gitlab/centurion/./app.py”, line 14, in read_root r = await request.body() File “/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/starlette/requests.py”, line 234, in body async for chunk in self.stream(): File “/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/starlette/requests.py”, line 215, in stream raise RuntimeError(“Stream consumed”) RuntimeError: Stream consumed

[hasier]

Considering this has been the way starlette has worked for a while now, maybe we could consider the Sentry SDK has “misused” form() and get the fix done here? (To be clear, I do feel starlette has a bit of an inconsistent API for these attributes so it’d be good to clarify with them in https://github.com/encode/starlette/discussions/1933 if it’s actually expected and straighten it out, but it’s also true that form() has worked like this for a while) Also, many people using Sentry won’t be able to get into the latest Starlette version straight away - mainly because a lot of people using FastAPI depend on its pin on Starlette-, so adding the fix here and marking the version 1.9.10 as yanked would make it immediately work for everyone, as well as unblocking further updates of the Sentry SDK library seeing as how people are downgrading to get around the issue.

I am thinking maybe it’d be an option to work around the caching issue by getting it consumed into body() first and introduce a test that breaks when Starlette has their issue fixed? The test could simply be 1) call request.form() 2) expect an exception calling request.body(). When that happens, the workaround can then be removed and maybe Sentry SDK could specify the minimum pin to use Starlette so that it goes hand in hand with the fix. wdyt?

# sentry_sdk/integrations/starlette.py
from starlette.datastructures import FormData
from starlette.formparsers import FormParser, MultiPartParser

async def form(self):
    # type: (StarletteRequestExtractor) -> typing.Optional[typing.Mapping[str, typing.Any]]
    if multipart is None:
        return None

    # Parse the body first to get it cached, as Starlette does not cache form() as it
    # does with body() and json() https://github.com/encode/starlette/discussions/1933
    await self.request.body()
    return await self.request.form()

EDIT: PR by @antonpirker following this approach https://github.com/getsentry/sentry-python/pull/1724 Thanks! 🙏

[hasier]

I think this issue can be marked as solved, as it’s been fixed in version 1.11.0 (it’s working for me at least). Thanks for your work @antonpirker! 🙇