question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Any interest in an UnixHTTPConnectionPool?

See original GitHub issue

I’m happy to create a PR, but it seems like a pretty common use case to want to establish an HTTP connection over a unix socket.

I see this abstraction happening more commonly through requests, but was a little surprised that this doesn’t just exist as a connection within urllib3.

The code I’ve thrown together (not yet battle tested) is pretty trivial to do when you get rid of the requests bits, and it’s boiled down to just:

class UnixHTTPConnection(HTTPConnection):
    def __init__(self, host, **kwargs):
        # We're using the `host` as the socket path, but
        # urllib3 uses this host as the Host header by default.
        # If we send along the socket path as a Host header, this is
        # never what you want and would typically be malformed value.
        # So we fake this by sending along `localhost` by default as
        # other libraries do.
        self.socket_path = host
        super(UnixHTTPConnection, self).__init__(host='localhost', **kwargs)

    def _new_conn(self):
        sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
        if self.timeout is not socket._GLOBAL_DEFAULT_TIMEOUT:
            sock.settimeout(self.timeout)
        sock.connect(self.socket_path)
        return sock


class UnixHTTPConnectionPool(HTTPConnectionPool):
    ConnectionCls = UnixHTTPConnection

If this is something you’d be interested in pulling into urllib3 as a core connection pool, I think this would be great to have, and I’ll convert this into a PR. If not, would be curious if this seems outside the scope of urllib3 or some other reason.

The sources of this off the top of my head are within docker-py, which again, uses this sorta abstraction except a bit heavy handed since it also needs the requests adapter https://github.com/docker/docker-py/blob/master/docker/transport/unixconn.py. Then requests-unixsocket project https://github.com/msabramo/requests-unixsocket.

And again, nothing explicitly for those of us that like just directly using urllib3.

I plan on adding this into Sentry very soon, and will cut over to using this for some of our critical internal services to get it battle tested.

I also noticed that this is sorta being used within test suite: https://github.com/urllib3/urllib3/blob/0f85e05af9ef2ded671a7b47506dfd24b32decf0/test/test_connectionpool.py#L35-L45 I’m not entirely sure of the context here, but it’s definitely just used for some mock class. Seems relevant though. 😃

Issue Analytics

  • State:open
  • Created 5 years ago
  • Comments:21 (21 by maintainers)

github_iconTop GitHub Comments

3reactions
theacodescommented, Oct 31, 2018

since requests is our primary downstream user, let’s match their scheme. It’s also a scheme that makes sense, nginx’s just looks… weird.

I’m fine with UnixHTTPConnection being in contrib.

1reaction
sigmavirus24commented, Oct 29, 2018

Yeah. I think we just want to be explicit in the documentation (like a .. warning at the top) that this does not work with PoolManager. That should help deflect most issues.

Read more comments on GitHub >

github_iconTop Results From Across the Web

`docker-compose up` times out with UnixHTTPConnectionPool
And I'm still trying to figure out whether or not there is a correlation with our agent's resources being at full use. docker...
Read more >
hangs on stopping unms-netflow - Ubiquiti Community
ERROR: for unms UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=70). ERROR: An HTTP request took too long to complete.
Read more >
trunk-recorder/Lobby - Gitter
Any interest in either? sally-yachts. @sally-yachts. Jan 24 09:13.
Read more >
Full Text Bug Listing - Red Hat Bugzilla
ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. ... If the client is interested only in a single stats instance, ...
Read more >
Running Docker CE on OSX High Sierra - Docker is timing out ...
Not sure if theres anything I can do... or ? if there is? any thoughts? Thank you! ... Interested in gaining a new...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found