uvicorn not closing sockets

We are using the latest uvicorn 0.15. It seems that after we upgraded to uvicorn 0.15 that our AWS load balancer health checks fail after a certain amount of time. Upon further investigation we found that all of our sockets are being used up because they are staying open in a CLOSE_WAIT status.

# netstat -ant | awk '{print $6}' | sort | uniq -c | sort -n
   1 Foreign
   1 LISTEN
   1 established)
   7 TIME_WAIT
  13 ESTABLISHED
7047 CLOSE_WAIT

Workaround

For now we have switched our worker from UvicornWorker to UvicornH11Worker and it has resolved our issue.

Environment

Running uvicorn 0.15.0 with CPython 3.7.12 on Linux

[shahar-medigate]

We encountered the issue in our systems as well, a simple fastapi service behind AWS EKS load-balancer we also managed to reproduce the issue to a single commit https://github.com/encode/uvicorn/commit/960d4650db0259b64bc41f69bc7cdcdb1fdbcbf3

A simple program to reproduce

import uvicorn

async def app(scope, receive, send):
    assert scope["type"] == "http"

    await send(
        {
            "type": "http.response.start",
            "status": 200,
            "headers": [
                [b"content-type", b"text/plain"],
            ],
        }
    )
    await send(
        {
            "type": "http.response.body",
            "body": b"Hello, world!",
        }
    )


if __name__ == "__main__":
    uvicorn.run(app, host="0.0.0.0", reload=False, log_level=5)

and run nc -z localhost:8000 now you will have a new CLOSE_WAIT connection with netstat

apparently, it got something to do with uvloop don’t call connection_lost or something we bypass the issue by setting loop="asyncio" to the run function

[pacoyang]

I don’t see how the change of worker class would solve this issue 🤔

It maybe because h11 will close the transport when if the connection is invalid HTTP request, this issue similarity to #1192 https://github.com/encode/uvicorn/blob/master/uvicorn/protocols/http/h11_impl.py#L140