question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Session does not consistently respects proxy environment variables.

See original GitHub issue

There are inconsistencies when working with proxy environment variables using Session objects.

When using Session#request or verb methods (get, post, put), those work correctly and use the proxy when they are present.

However Session#send(PreparedRequest) does not respect the variables by default, but does when it handles redirect responses, e.g. 303.

I’ve written a test class to present this behavior.

Expected Result

All methods in Session to respect the proxy variable.

Actual Result

Session#send(PreparedRequest) does not respect proxy environment variables.

Reproduction Steps

Test case 1 fails with a successful request.

import requests
import pytest

# Copied from requests/tests/utils.py for making this a standalone
import os
import contextlib

@contextlib.contextmanager
def override_environ(**kwargs):
    save_env = dict(os.environ)
    for key, value in kwargs.items():
        if value is None:
            del os.environ[key]
        else:
            os.environ[key] = value
    try:
        yield
    finally:
        os.environ.clear()
        os.environ.update(save_env)

class TestSession:

    def test_respect_proxy_env_on_send(self):
        session = requests.Session()

        # Does not respect the proxy configuration and ignores the proxy
        request = requests.Request(
            method='GET', url='https://www.google.com/'
        )

        with override_environ(https_proxy='http://example.com'):
            try:
                session.send(request.prepare())
                assert False, "The proxy is invalid this request should not be successful."
            except requests.exceptions.ProxyError as e:
                assert 'Cannot connect to proxy' in str(e)
        
    def test_respect_proxy_env_on_send_with_redirects(self):
        session = requests.Session()

        # Returns a 303 to www.google.com and respects the proxy
        request = requests.Request(
            method='GET', url='https://google.com/'
        )

        with override_environ(https_proxy='http://example.com'):
            try:
                session.send(request.prepare())
                assert False, "The proxy is invalid this request should not be successful."
            except requests.exceptions.ProxyError as e:
                assert 'Cannot connect to proxy' in str(e)

    def test_respect_proxy_env_on_get(self):
        session = requests.Session()

        with override_environ(https_proxy='http://example.com'):
            try:
                # No redirects
                session.get('https://www.google.com')
                assert False, "The proxy is invalid this request should not be successful."
            except requests.exceptions.ProxyError as e:
                assert 'Cannot connect to proxy' in str(e)

    def test_respect_proxy_env_on_request(self):
        session = requests.Session()

        with override_environ(https_proxy='http://example.com'):
            try:
                # No redirects
                session.request(method='GET', url='https://www.google.com')
                assert False, "The proxy is invalid this request should not be successful."
            except requests.exceptions.ProxyError as e:
                assert 'Cannot connect to proxy' in str(e)

System Information

$ python -m requests.help
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.10"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.8.1"
  },
  "platform": {
    "release": "5.9.8-100.fc32.x86_64",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.25.0"
  },
  "system_ssl": {
    "version": "100020ff"
  },
  "urllib3": {
    "version": "1.26.2"
  },
  "using_pyopenssl": false
}

Issue Analytics

  • State:open
  • Created 3 years ago
  • Reactions:1
  • Comments:6 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
tscheburaschkacommented, Jan 28, 2021

Hello, I would like to add to the topic of this issue. As far as I can see, the hierarchy of proxy-settings applied to the final request is not consistent or at least not as I would expect. From my understanding, the documentation suggests the following hierarchy of settings (right supersedes left): environment variables < session settings < method call kwargs During usage I found that the actual hierarchy implemented is (only for proxy settings): session settings < environment variables < method call kwargs Is this intended behavior? If so, I would suggest to clarify the respective documentation.

0reactions
tscheburaschkacommented, Jan 28, 2021

Concerning the documentation, I read the section Proxies as if the environment configuration is “overridden” by settings in Python code, and the examples shown suggest the hierarchy I indicated above.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Update Process Using Curl Does Not Access the ... - SUSE
If these variables are not set in curl's session environment, curl will not use the proxy server and updates will not occur as...
Read more >
Configuring Proxy Settings for All Apps | Pivotal Docs
To globally configure proxy settings for Pivotal Platform apps, you must set three environment variables for both the staging environment ...
Read more >
Using a custom proxy - Postman Learning Center
Prerequisites. Setting up Postman. Additional Resources. Blog Posts. Capture Responses Using the Postman Proxy. Postman Newsletter
Read more >
System Proxy Configuration - Cribl Docs
The environment variables' names can be either uppercase or lowercase. However, if you set duplicate versions of the same name, the lowercase version...
Read more >
Configuration - ShinyProxy
When a user is authenticated, the following environment variables will be available in any Shiny application launched by the user: SHINYPROXY_USERNAME : the ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found