(request/feature) Transparent TLSv1.2 websocket analysis not working

Is your feature request related to a problem? Please describe.

Yes it is. I am trying to DE TLSv1.2 packets of a game so that I can make those API calls outside of the game itself, for statistical analysis. I’ve used Wireshark and mitmproxy to determine that the game is requesting a websocket upgrade, receives an HTTP 101 upgrade from the server after which all traffic is over said websocket. Mitmproxy doesn’t display this, and I cannot get certificates working with mitmdump despite even with this issue: https://github.com/mitmproxy/mitmproxy/issues/899#issuecomment-608996389

The result is always Client Handshake failed. The client may not trust the proxy's certificate. Note, the certificates are fine when using mitmproxy for the initial websocket upgrade so I don’t understand.

Describe the solution you’d like

Viewing websocket data transparently within mitmproxy or instructions on how to dump decrypted websocket data with mitmdump, as I cannot get certificates to work.

Describe alternatives you’ve considered

I’ve tried every single library I can find for MiTM proxy self-attacks and none work, almost all are designed for browsers which this application is not and uses no part of. I’ve also tried MITMPROXY_SSLKEYLOGFILE but of course this doesn’t do anything since it refuses the certificate to start the handshake, as well as SSLKEYLOGFILE and LD_PRELOAD which as far as I know are also browser specific.

Additional context

My mitmdump command is as follows (foobar.py is from the issue aforementioned)

mitmdump -s ./foobar.py -p 8081 --cert *=~/.mitmproxy/mitmproxy-ca-cert.pem --set block_global=false --set keep_host_header=true

[danghuong18]

We have two separate issues here:

First, there is a connection from :49052 which is intercepted just fine. However, the upstream server’s certificate is not trusted, which is why mitmproxy does not continue:

192.168.1.139:49052: GET https://37.187.27.57:60442/
    Host: master.shirogames.com
    Upgrade: websocket
    Connection: Upgrade
    Sec-WebSocket-Key: SjNN5gbKf+FHXSzpc/3GUQ==
    Sec-WebSocket-Version: 13
    X-Ident: northgard
    X-Pass: 5b69ae0b2864a49d753b27fe974c36a485d605ca
 << Certificate verification error for master.shirogames.com: certificate signature failure (errno: 7, depth: 0)

The fix here is to set mitmproxy’s ssl_insecure option to true.

Second, there is a connection from :40521 which we can’t intercept:

192.168.1.139:40521: Client Handshake failed. The client may not trust the proxy's certificate for master.shirogames.com.
192.168.1.139:40521: ClientHandshakeException('Cannot establish TLS with client (sni: master.shirogames.com): TlsException("SSL handshake error: Error([(\'SSL routines\', \'ssl3_read_bytes\', \'tlsv1 alert decrypt error\')])")')

This likely is certificate pinning.

I cant capture websockets, could you help me? I tried everything but it still didn’t work.

My version mitmproxy: Mitmproxy: 5.1.1 Python: 3.8.2 OpenSSL: OpenSSL 1.1.1g 21 Apr 2020 Platform: Windows-10-10.0.18362-SP0

Thanks.

[mhils]

We have two separate issues here:

First, there is a connection from :49052 which is intercepted just fine. However, the upstream server’s certificate is not trusted, which is why mitmproxy does not continue:

192.168.1.139:49052: GET https://37.187.27.57:60442/
    Host: master.shirogames.com
    Upgrade: websocket
    Connection: Upgrade
    Sec-WebSocket-Key: SjNN5gbKf+FHXSzpc/3GUQ==
    Sec-WebSocket-Version: 13
    X-Ident: northgard
    X-Pass: 5b69ae0b2864a49d753b27fe974c36a485d605ca
 << Certificate verification error for master.shirogames.com: certificate signature failure (errno: 7, depth: 0)

The fix here is to set mitmproxy’s ssl_insecure option to true.

Second, there is a connection from :40521 which we can’t intercept:

192.168.1.139:40521: Client Handshake failed. The client may not trust the proxy's certificate for master.shirogames.com.
192.168.1.139:40521: ClientHandshakeException('Cannot establish TLS with client (sni: master.shirogames.com): TlsException("SSL handshake error: Error([(\'SSL routines\', \'ssl3_read_bytes\', \'tlsv1 alert decrypt error\')])")')

This likely is certificate pinning.