WebSocket headers encoding problem

Checklist

• The bug is reproducible against the latest release and/or master.
• There are no similar issues or pull requests to fix it yet.

Describe the bug

If there is a websocket connection with a non-ascii char in header, the server will explode. The problem seems to be from here: https://github.com/encode/uvicorn/blob/9dc5a43209fe081ba3e74135189252bfddf75587/uvicorn/protocols/websockets/websockets_impl.py#L102 Also, I checked the ASGI doc and in section 2.2.5 it states:

• These are byte strings of the exact byte sequences sent by the client/to be sent by the server. While modern HTTP standards say that headers should be ASCII, older ones did not and allowed a wider range of characters. Frameworks/applications should decode headers as they deem appropriate.

So, I think uvicorn should allow for non-ascii chars in header. It is definitely the only reason that we can’t use uvicorn in production yet.

To reproduce

Expected behavior

Doesn’t explode.

Actual behavior

Explodes with an exception.

Debugging material

Environment

• OS / Python / Uvicorn version: just run uvicorn --version
• The exact command you’re running uvicorn with, all flags you passed included. If you run it with gunicorn please do the same. If there is a reverse-proxy involved and you cannot reproduce without it please give the minimal config of it to reproduce.

Additional context

[tomchristie]

@lavandosovich - I’d be happy to spend a little time thinking about this if we start from really basic first principles.

Can you give an example of an HTTP response that you’d like to be able to send, that you currently can’t because of this limitation?

[Kludex]

@tomchristie since websockets is decoding using surrogateescape: https://github.com/aaugustin/websockets/blob/3401751d4f1200d398c5e69f9c4a0ece862ff36b/src/websockets/legacy/http.py#L169

We can encode with surrogateescape, and reopen https://github.com/encode/uvicorn/pull/1005. Wdyt?