Feature request: mechanism to set TCP socket connect timeout

Thanks for mitmproxy! It’s awesome.

I’ve spent a few days debugging an issue in my environment (a custom Selenium grid that runs Firefox and proxies connections through mitmproxy, uses inline scripts, and has specific requirements for limiting the time that things can take). I believe that having the ability to set TCP socket connection timeouts would solve some specific issues for me. I’ve distilled it down here to some curl commands, but be aware it’s not the real environment.

Steps to reproduce the problem:

1. Start mitmdump --port 8088
2. Run time curl --proxy http://localhost:8088 http://169.254.111.111/fake, notice it hangs and then takes real 1m15.650s before raising a 502 and timing out (the actual time is system TCP config dependent).

What is the expected behavior?

I’d like the ability to make this timeout after X seconds. Specifically, I’d like to set a TCP connection timeout on the socket.socket object, after socket creation but before connect.

What went wrong?

As far as I can tell, there’s no way to do it currently.

https://github.com/mitmproxy/mitmproxy/blob/0.17.x/netlib/tcp.py#L699

A few roadblocks exist here to solving this myself:

• The creation of the socket is relatively deeply nested and there are not hooks to configure it.
• You cannot use the serverconnect hook, it fires before the socket object is actually created.
• You cannot use the TCPClient.settimeout method for the same reason, the timeout needs to be set before connect but after the socket is created.
• You cannot use Python’s native socket.setdefaulttimeout in an inline script because it still breaks SSL connections in the same way as this old, closed issue (and this might actually be the wrong fix anyway, since it’s a full socket timeout not just a connect timeout).

I basically want a productionized way to do this patch:

index 8e46d4f..8e04138 100644
--- a/netlib/tcp.py
+++ b/netlib/tcp.py
@@ -648,9 +648,12 @@ class TCPClient(_Connection):
     def connect(self):
         try:
             connection = socket.socket(self.address.family, socket.SOCK_STREAM)
+            connection.settimeout(CONNECT_TIMEOUT_SECONDS)
             if self.source_address:
                 connection.bind(self.source_address())
             connection.connect(self.address())
+            # Reset socket timeout to system defaults because the connection is now established.
+            connection.settimeout(None)
             if not self.source_address:
                 self.source_address = Address(connection.getsockname())
             self.rfile = Reader(connection.makefile('rb', self.rbufsize))

I guess this could be either through a new hook like beforeserverconnect for inline scripts, or a command line flag, etc. It might also be nice to pull out the socket.socket creation to it’s own method that can be more easily manipulated (I’ve ended up having to monkey-patch the whole connect method unfortunately).

Any other comments?

I think this would fix https://github.com/mitmproxy/mitmproxy/issues/1044

---

Mitmproxy Version: v0.16 (but relates to master) Operating System: Linux and Mac

[synchronizing]

Any updates on the above?

[mhils]

We thankfully don’t use socket.makefile anymore, so setting socket timeouts should be relatively easy now. The relevant call to open upstream connections is happening here: https://github.com/mitmproxy/mitmproxy/blob/1e9707307815144a3106c3b5056111971ad07c6d/mitmproxy/proxy/server.py#L139-L152

The most useful approach would probably be to manually create a socket.socket object before all that, add it to ServerConnectionHookData to expose it to addons in the server_connect hook (we now have separate server_connect and server_connected hooks, the first one fires early enough), and then use it in the asyncio.open_connection call. This would provide addons with full control over the socket object to do all sorts of shenanigans (such as #436).