WebSockets support.

Currently httpx is squarely focused on HTTP’s traditional request / response paradigm, and there are well-established packages for WebSocket support such as websockets. In an HTTP/1.1-only world, this split of responsabilities makes perfect sense as HTTP requests / WebSockets work independently.

However, with HTTP/2 already widely deployed and HTTP/3 standardisation well under way I’m not sure the model holds up:

• implementations such as websockets are usually tied to HTTP/1.1 only, whereas httpx has support for HTTP/2 (and hopefully soon HTTP/3)
• we are missing out on the opportunity to multiplex “regular” HTTP request with WebSocket connections

Using the sans-IO wsproto combined with httpx’s connection management we could provide WebSocket support spanning HTTP/1.1, HTTP/2 and HTTP/3. What are your thoughts on this?

One caveat: providing WebSocket support would only make sense using the AsyncClient interface.

[tomchristie]

We’ve actually got some neat stuff we can do around this now, but it needs documenting. Going to keep this open to track that.

See the stream extension in the httpcore docs for a rough outline.

[tomchristie]

So my initial take on this is that we’d want to first expose a connection upgrade API, and then built websockets support over that, so something like…

def request(...) -> ...  #  Our standard request/response interface at the Transport API level.
def connect(...) -> Connection  # New method at the Transport API level, exposing raw connection capabilities.

Then at the client level, have websocket support, that uses connect under the hood.

async with client.websocket(...) as ws:
    await ws.send(...)
    message = await ws.receive()

There’s a couple of potential awkwardnesses about that tho…

• It pushes the websocket networking logic into httpx, rather than having it in httpcore.
• It makes it awkward to implement alternate websocket backends, eg. ASGITransport would need to de-marshall the bytes on the connection back into websocket events, rather than “seeing” an API abstraction that’s at the level of the events themselves.
• The connection API is possibly more involved than the websocket API, and tougher for us to home in on.

So, after a bit more thinking, I reckon we should instead aim to provide a Transport-level interface specifically for websockets.

def request(...) -> ...  #  Our standard request/response interface at the Transport API level.
def websocket(...) -> WebSocket  # New method at the Transport API level, exposing websocket capabilities.

This wouldn’t preclude us possibly also adding a raw “just give me the connection” level abstraction too at some other point, in order to provide for low-level CONNECT, Upgrade and HTTP/2 bi-directional streaming support. But we could treat that as a lower priority, depending on if we’re actually seeing any demand/use-cases for exposing those capabilities.

def request(...) -> ...  #  Our standard request/response interface at the Transport API level.
def websocket(...) -> WebSocket  # New method at the Transport API level, exposing websocket capabilities.
def connect(...) -> Connection  # New method at the Transport API level, exposing raw CONNECT/Upgrade/HTTP2 data streaming capabilities.

The information returned from the low-level websocket method would need to be all the standard response code/headers stuff, plus an interface that just exposes something like send_event(), receive_event() and close().

There’s also the question of what the API ought to look like at the httpx level. One thing that tends to be a bit fiddly here is that websockets can return either bytes or str frames, but we’d still like our users to be able to access the data in a nice type-checked fashion. Rather than expose multiple kinds of send/receive methods, we might be able to do with just these, by…

1. Having a form like ws.send(text=...) to distinguish on sending.
2. Having a form like msg = ws.receive(); print(msg.text) to distinguish on receiving. The .text property could strictly return str, and could raise an error if the received data frame was actually in the unexpected binary mode.
3. For convenience we probably want a json argument in both case. We could default to handling JSON over text frames. Optionally we might include a flag on the client.websocket() method setting the expected mode to either text or binary, and using that for the JSON framing, plus erroring out if the incorrect style is sent/received anytime.

def send(*, text: str = None, content: bytes = None, json: typing.Any = None)
def receive()  # Returns an object with `.text`, `.content`, `.json`, `.payload [str|bytes]`
def close()

We can transparently deal with ping/pong frames during any .send()/receive(), and context managed async transports could also send background pings.

We might well also want .iter_bytes(), .iter_text(), and .iter_json() methods, for convenience.

I’m aware this is all in a bit of a “jotting down” style, please do feel free to let me know if I’m not being clear enough about anything here.