HTTP CONNECT request sent in different packets while using an HTTPS Proxy

Note: At the request of the requests team, I’ve opened a ticket here.

When using an HTTPS proxy, HTTP headers (and other parts of the request) are sent at different times then the initial HTTP CONNECT method, causing some proxy services to abort the connection.

Expected Result

The entire HTTP request should be sent in the same packet as the CONNECT method.

Actual Result

HTTP Headers and trailing \r\n are sent at a different time.

You can see this in the Wireshark Capture:

First Packet contains the HTTP CONNECT (from here)

Second packet shows the left over \r\n (from here)

This results in some proxy servers aborting the connection.

Reproduction Steps

>>> import requests
>>> proxies = { 'https': 'https://someproxy:443' }
>>> requests.get('https://www.google.com', proxies=proxies)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/user/Library/Python/2.7/lib/python/site-packages/requests/api.py", line 70, in get
    return request('get', url, params=params, **kwargs)
  File "/Users/user/Library/Python/2.7/lib/python/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/user/Library/Python/2.7/lib/python/site-packages/requests/sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/user/Library/Python/2.7/lib/python/site-packages/requests/sessions.py", line 596, in send
    r = adapter.send(request, **kwargs)
  File "/Users/user/Library/Python/2.7/lib/python/site-packages/requests/adapters.py", line 485, in send
    raise ProxyError(e, request=request)
requests.exceptions.ProxyError: HTTPSConnectionPool(host='www.google.com', port=443): Max retries exceeded with url: / (Caused by ProxyError('Cannot connect to proxy.', error(54, 'Connection reset by peer')))

System Information

$ python -V
Python 2.7.11

(Same behavior on Python 3)

MacOSX

Work Around

The following work around sends the entire HTTP CONNECT request at the same time:

https://gist.github.com/brad-anton/ff517025a86754793b680b3c5cfc9797

[brad-anton]

One other thing @SethMichaelLarson - wouldnt the fix be in here rather then in urllib?

• https://github.com/python/cpython/blob/master/Lib/http/client.py#L895-L898

[sethmlarson]

This was fixed in https://github.com/python/cpython/pull/24780 and is available in Python 3.10+