PUSH_PROMISE on closed stream

We are testing @mitmproxy against http://http2-push.io, which sends pushed two streams if you request the / path. This works fine from curl and a simple h2 client.

However as soon as you try to browse to that page from Firefox things start be get weird and broken. Browsing there with a fresh session works once, and then the PUSH_PROMISE frames will fail or get swallowed.

It turns out, that if you hit the client cache for /, the server responds with a 304, but still sends the PUSH_PROMISE frames. However, at that point the original stream is already closed, which cases hyper-h2 to yield the following error message: h2.exceptions.ProtocolError: Attempted to push on closed stream.

According to RFC 7540 Section 6.6:

PUSH_PROMISE frames MUST only be sent on a peer-initiated stream that is in either the “open” or “half-closed (remote)” state.

Looking at the received frames, we see:

HEADERS with stream_id=1,  code=304 and flags=(END_HEADERS and END_STREAM)
PUSH_PROMISE with stream_id=1, promised_stream_id=2, and a headers fragment
PUSH_PROMISE with stream_id=1, promised_stream_id=4, and a headers fragment

hyper-h2 now processes this in a way that stream 1 is closed before the PUSH_PROMISE is handled, therefore generating above error code - which is totally valid following the reasoning above.

This error does not happen if we have a clean cache, and therefore get the following frames from the server:

HEADERS with stream_id=1,  code=200 and flags=END_HEADERS
PUSH_PROMISE with stream_id=1, promised_stream_id=2, and a headers fragment
PUSH_PROMISE with stream_id=1, promised_stream_id=4, and a headers fragment
DATA with stream_id=1,  some data and flags=END_STREAM

Here is a minimal example using hyper-h2 as client to trigger that error:

import certifi
import h2.connection
import h2.events

import traceback
import errno
import socket
import ssl
import time

SERVER_NAME = 'http2-push.io'

socket.setdefaulttimeout(2)

c = h2.connection.H2Connection()
c.initiate_connection()

ctx = ssl.create_default_context(cafile=certifi.where())
ctx.set_alpn_protocols(['h2', 'http/1.1'])
ctx.set_npn_protocols(['h2', 'http/1.1'])

s = socket.create_connection((SERVER_NAME, 443))
s = ctx.wrap_socket(s, server_hostname=SERVER_NAME)

s.sendall(c.data_to_send())

headers = [
    (':method', 'GET'),
    (':path', '/'),
    (':authority', SERVER_NAME),
    (':scheme', 'https'),
    ('user-agent', 'custom-python-script'),
    # omitting this header to get a working push:
    ('cookie', "__cfduid=d5c239ad129872798300aad6d474b69021479849732; _ga=GA1.2.1109854913.1477608538"),
    (b'if-modified-since', b"Thu, 27 Oct 2016 11:02:40 GMT"),
]
c.send_headers(1, headers, end_stream=True)
s.sendall(c.data_to_send())


while True:
    data = s.recv(65536 * 256)
    if not data:
        break

    try:
        events = c.receive_data(data)
        s.sendall(c.data_to_send())
    except Exception as e:
        print(traceback.format_exc())
        break

    for event in events:
        print(event)

As far as we can tell at the moment, this seems to be a bug in whatever webserver software http2-push.io is using. Looking at the response headers, it looks like: ('server', 'cloudflare-nginx')

[ghedo]

@Lukasa @Kriechi we are going to deploy a fix for this next week if nothing gets in the way.

In the future feel free to CC me if you see other h2-related bugs on Cloudflare, so we can cut the middle-men and get the information to the right people more quickly 😃

[Kriechi]

This seems to be resolved now - at least my example from above does not throw an error any more. Thanks @ghedo for the fast fix!