reverse proxy mode for h2→h2c connections not working

Steps to reproduce the problem:

1. Start listening on local port 8081 like this: nc -vl 8081
2. Start mitmproxy like this: mitmdump --http2 --reverse http://localhost:8081
3. Use nghttp to make a request like this: nghttp -v https://localhost:8080/
4. mitmdump prints:

('::ffff:127.0.0.1', 39950, 0, 0): clientconnect
('::ffff:127.0.0.1', 39950, 0, 0): Traceback (most recent call last):
  File "/home/vasiliy/tmp/env8/lib/python3.5/site-packages/mitmproxy/proxy/server.py", line 118, in handle
    root_layer()
  File "/home/vasiliy/tmp/env8/lib/python3.5/site-packages/mitmproxy/proxy/modes/reverse_proxy.py", line 13, in __call__
    layer()
  File "/home/vasiliy/tmp/env8/lib/python3.5/site-packages/mitmproxy/proxy/protocol/tls.py", line 383, in __call__
    layer()
  File "/home/vasiliy/tmp/env8/lib/python3.5/site-packages/mitmproxy/proxy/protocol/http2.py", line 325, in __call__
    self._initiate_server_conn()
  File "/home/vasiliy/tmp/env8/lib/python3.5/site-packages/mitmproxy/proxy/protocol/http2.py", line 111, in _initiate_server_conn
    self.connections[self.server_conn].initiate_connection()
KeyError: <ServerConnection: localhost:8081>

Traceback (most recent call last):
  File "/home/vasiliy/tmp/env8/lib/python3.5/site-packages/mitmproxy/proxy/server.py", line 118, in handle
    root_layer()
  File "/home/vasiliy/tmp/env8/lib/python3.5/site-packages/mitmproxy/proxy/modes/reverse_proxy.py", line 13, in __call__
    layer()
  File "/home/vasiliy/tmp/env8/lib/python3.5/site-packages/mitmproxy/proxy/protocol/tls.py", line 383, in __call__
    layer()
  File "/home/vasiliy/tmp/env8/lib/python3.5/site-packages/mitmproxy/proxy/protocol/http2.py", line 325, in __call__
    self._initiate_server_conn()
  File "/home/vasiliy/tmp/env8/lib/python3.5/site-packages/mitmproxy/proxy/protocol/http2.py", line 111, in _initiate_server_conn
    self.connections[self.server_conn].initiate_connection()
KeyError: <ServerConnection: localhost:8081>

mitmproxy has crashed!
Please lodge a bug report at: https://github.com/mitmproxy/mitmproxy
('::ffff:127.0.0.1', 39950, 0, 0): clientdisconnect

Meanwhile, nghttp prints that it has sent a bunch of HTTP/2 frames, but not received any.

Meanwhile, no connections are made to port 8081.

Any other comments? What have you tried so far?

I’m not sure what I would expect to happen. What I would like to happen is that mitmproxy talk cleartext HTTP/2 (direct or upgrading) to port 8081, and talk HTTP/2-over-TLS to nghttp.

If I use a Web browser instead of nghttp, then mitmproxy talks cleartext HTTP/1.1 to port 8081, and talks HTTP/1.1-over-TLS to the browser.

See also #2116.

System information

(this is mitmproxy master)

Mitmproxy version: 3.0.0 (release version) Python version: 3.5.2 Platform: Linux-4.4.0-65-generic-x86_64-with-Ubuntu-16.04-xenial SSL version: OpenSSL 1.0.2g 1 Mar 2016 Linux distro: Ubuntu 16.04 xenial

[vfaronov]

I took the liberty to edit the title, because reverse proxy mode for h2→h2 definitely does work for me. It’s only when I try to do strange things that I get strange results 😃

[TuomasKiviaho]

However I think this would probably make the h2-internals in mitmproxy overly complex for a rarely used feature (I guess?)

I was trying to do similar TLS offloading like what HAProxy is capable of providing to Jetty (https://www.eclipse.org/jetty/documentation/9.3.x/http2-enabling.html and https://www.eclipse.org/jetty/documentation/9.4.x/http2-configuring-haproxy.html), but apparently that’s not possible.

@mhils suggested force_alpn at https://discourse.mitmproxy.org/t/using-mitmproxy-to-terminate-tls-for-http-2/404 and that still puzzles me. Would it be possible to make TLS offloading possible (alpn+h2) without even needing h2c.