question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

http 204 causes gunicorn shutdown on appengine [BUG]

See original GitHub issue

Describe the bug

I’ve found that a simple endpoint with status_code=204 is causing gunicorn to shutdown when running on Google appengine.

I suspect the bug is probably outside of FastAPI (eg maybe in uvicorn?), but I haven’t yet managed to reproduce it with a simpler stack, and maybe this will help others with the same issue.

To Reproduce

  1. Create the following:

main.py:

from fastapi import FastAPI

app = FastAPI()


@app.get("/")
def read_root():
    return {"Hello": "World"}


@app.delete("/", status_code=204)
def delete_root():
    print("deleted a thing...")
    return ""

requirements.txt:

fastapi
gunicorn
uvicorn

app.yaml:

runtime: python37

instance_class: F1

entrypoint: gunicorn --bind :$PORT --workers 1 --worker-class uvicorn.workers.UvicornWorker main:app

handlers:
    - url: /.*
      secure: always
      script: auto
  1. Install gcloud cli, login to gcloud and then deploy to appengine with:
gcloud app deploy
  1. Call the delete endpoint
curl -XDELETE https://YOURPROJECT.appspot.com
  1. It hangs for a couple of seconds and then returns an error like this:
<html><head>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<title>500 Server Error</title>
</head>
<body text=#000000 bgcolor=#ffffff>
<h1>Error: Server Error</h1>
<h2>The server encountered an error and could not complete your request.<p>Please try again in 30 seconds.</h2>
<h2></h2>
</body></html>

And logs as follows:

2020-05-23 15:08:11.658 NZST deleted a thing...
2020-05-23 15:08:13.193 NZST [start] 2020/05/23 03:08:13.192900 Quitting on terminated signal
2020-05-23 15:08:13.198 NZST [2020-05-23 03:08:13 +0000] [7] [INFO] Handling signal: term
2020-05-23 15:08:13.259 NZST [2020-05-23 03:08:13 +0000] [15] [INFO] Shutting down
2020-05-23 15:08:13.259 NZST [2020-05-23 03:08:13 +0000] [15] [INFO] Error while closing socket [Errno 9] Bad file descriptor
2020-05-23 15:08:13.360 NZST [2020-05-23 03:08:13 +0000] [15] [INFO] Waiting for application shutdown.
2020-05-23 15:08:13.360 NZST [2020-05-23 03:08:13 +0000] [15] [INFO] Application shutdown complete.
2020-05-23 15:08:13.360 NZST [2020-05-23 03:08:13 +0000] [15] [INFO] Finished server process 
2020-05-23 15:08:13.361 NZST [2020-05-23 03:08:13 +0000] [15] [INFO] Worker exiting (pid: 15)
2020-05-23 15:08:13.502 NZST [2020-05-23 03:08:13 +0000] [7] [INFO] Shutting down: Master
2020-05-23 15:08:13.556 NZST [start] 2020/05/23 03:08:13.555603 Start program failed: termination triggered by nginx exit
  1. But I expected it to return an empty body with http status 204.

Note - changing status code to something else, eg 200 resolves the issue.

Expected behavior

I expected an empty 204 response to be returned.

Environment

  • OS: Linux (appengine)

  • FastAPI Version

0.54.2

  • Python version

3.7 (whatever as used by appengine)

See also

Cross-post on the appengine issue tracker https://issuetracker.google.com/issues/157312477

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Reactions:3
  • Comments:6 (1 by maintainers)

github_iconTop GitHub Comments

1reaction
jwmngcommented, Aug 24, 2020

Can confirm this happens on standalone uvicorn 0.11.8 as well. The 204 response indicates that no content will follow. However, return "" sends a response body that is two characters long for some reason. This causes an ASGI exception, the related uvicorn traceback shows:

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "<snip>\env\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 388, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "<snip>\env\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 45, in __call__
    return await self.app(scope, receive, send)
  File "<snip>\env\lib\site-packages\fastapi\applications.py", line 179, in __call__
    await super().__call__(scope, receive, send)
  File "<snip>\env\lib\site-packages\starlette\applications.py", line 111, in __call__
    await self.middleware_stack(scope, receive, send)
  File "<snip>\env\lib\site-packages\starlette\middleware\errors.py", line 181, in __call__
    raise exc from None
  File "<snip>\env\lib\site-packages\starlette\middleware\errors.py", line 159, in __call__
    await self.app(scope, receive, _send)
  File "<snip>\env\lib\site-packages\starlette\exceptions.py", line 82, in __call__
    raise exc from None
  File "<snip>\env\lib\site-packages\starlette\exceptions.py", line 71, in __call__
    await self.app(scope, receive, sender)
  File "<snip>\env\lib\site-packages\starlette\routing.py", line 566, in __call__
    await route.handle(scope, receive, send)
  File "<snip>\env\lib\site-packages\starlette\routing.py", line 227, in handle
    await self.app(scope, receive, send)
  File "<snip>\env\lib\site-packages\starlette\routing.py", line 44, in app
    await response(scope, receive, send)
  File "<snip>\env\lib\site-packages\starlette\responses.py", line 139, in __call__
    await send({"type": "http.response.body", "body": self.body})
  File "<snip>\env\lib\site-packages\starlette\exceptions.py", line 68, in sender
    await send(message)
  File "<snip>\env\lib\site-packages\starlette\middleware\errors.py", line 156, in _send
    await send(message)
  File "<snip>\env\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 482, in send
    output = self.conn.send(event)
  File "<snip>\env\lib\site-packages\h11\_connection.py", line 469, in send
    data_list = self.send_with_data_passthrough(event)
  File "<snip>\env\lib\site-packages\h11\_connection.py", line 502, in send_with_data_passthrough
    writer(event, data_list.append)
  File "<snip>\env\lib\site-packages\h11\_writers.py", line 77, in __call__
    self.send_data(event.data, write)
  File "<snip>\env\lib\site-packages\h11\_writers.py", line 97, in send_data
    raise LocalProtocolError("Too much data for declared Content-Length")
h11._util.LocalProtocolError: Too much data for declared Content-Length

If you return None the response body will remain empty and the 204 response works.

I’m not sure if this is a bug in FastAPI, Starlette or h11, or which of them interprets the empty string as a literal '' response

0reactions
therefromherecommented, Sep 4, 2022

Thanks @tiangolo , yes confirming this is fixed now

Read more comments on GitHub >

github_iconTop Results From Across the Web

Why are App Engine instances shutting down with a "start ...
Check if you return a status of 204 it can shut down your instance, It's a kind of bug in gunicorn, you can...
Read more >
I'm seeing 500 status and 204 error code. - Google Groups
A problem was encountered with the process that handled this request, causing it to exit. This is likely to cause a new process...
Read more >
subject:"\[google\-appengine\] error" - The Mail Archive
Hi , I have created a simple Node.js application and trying to deploy it to Google Cloud using command - gcloud app deploy...
Read more >
Tornado Documentation - Read the Docs
The Tornado web framework and HTTP server together offer a ... If the error was caused by an exception, an exc_info triple will...
Read more >
Contrast Documentation 3.8.9.5
Contrast applies bug fixes and develops new features on the latest ... Set the amount of time to run the agent before shutting...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found