pip does not respect proxy when downloading "wheel" as a dependency

Environment

• pip version: 18.0
• Python version: 3.5.6
• OS: linux

Platform: Xilinx Ultrascale+

Description

When using --proxy, pip normally downloads any required dependencies for the requested package without issue. But there is an edge condition when the dependency is “wheel”. In this case, the attempt to download errors out because pip is not going through the proxy. The workaround is to install wheel separately first. This was observed when trying to download and build scipy 1.4.1 from source.

Expected behavior

pip downloads wheel through the proxy.

How to Reproduce

1. From Linux on ARM, verify that wheel is not installed
2. pip install scipy --proxy <proxy>
3. pip errors out
4. pip install wheel --proxy <proxy>
5. pip install scipy --proxy <proxy>

It’s ok if the scipy installation errors out because you don’t have the compilers and required libraries setup. The error described here occurs before you reach that point.

Output

home# pip3 install scipy --proxy proxy1.lmco.com:80 --trusted-host pypi.org --trusted-host pypi.python.org --trusted-host files.pythonhosted.org Collecting scipy
  Downloading https://files.pythonhosted.org/packages/04/ab/e2eb3e3f90b9363040a3d885ccc5c79fe20c5b8a3caa8fe3bf47ff653260/scipy-1.4.1.tar.gz (24.6MB)
    100% |################################| 24.6MB 3.8MB/s
  Installing build dependencies ... error
  Complete output from command /usr/bin/python3 -m pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-d3vvzxn3 --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple --trusted-host pypi.org --trusted-host pypi.python.org --trusted-host files.pythonhosted.org -- wheel setuptools Cython>=0.29.13 "numpy==1.13.3; python_version=='3.5' and platform_system!='AIX'" "numpy==1.13.3; python_version=='3.6' and platform_system!='AIX'" "numpy==1.14.5; python_version=='3.7' and platform_system!='AIX'" "numpy==1.17.3; python_version>='3.8' and platform_system!='AIX'" "numpy==1.16.0; python_version=='3.5' and platform_system=='AIX'" "numpy==1.16.0; python_version=='3.6' and platform_system=='AIX'" "numpy==1.16.0; python_version=='3.7' and platform_system=='AIX'" "numpy==1.17.3; python_version>='3.8' and platform_system=='AIX'" pybind11>=2.4.0:
  Ignoring numpy: markers 'python_version == "3.6" and platform_system != "AIX"' don't match your environment
  Ignoring numpy: markers 'python_version == "3.7" and platform_system != "AIX"' don't match your environment
  Ignoring numpy: markers 'python_version >= "3.8" and platform_system != "AIX"' don't match your environment
  Ignoring numpy: markers 'python_version == "3.5" and platform_system == "AIX"' don't match your environment
  Ignoring numpy: markers 'python_version == "3.6" and platform_system == "AIX"' don't match your environment
  Ignoring numpy: markers 'python_version == "3.7" and platform_system == "AIX"' don't match your environment
  Ignoring numpy: markers 'python_version >= "3.8" and platform_system == "AIX"' don't match your environment
  Collecting wheel
    Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f92a75860>: Failed to establish a new connection: [Errno 101] Network is unreachable',)': /simple/wheel/
    Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f92a75710>: Failed to establish a new connection: [Errno 101] Network is unreachable',)': /simple/wheel/
    Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f92a75630>: Failed to establish a new connection: [Errno 101] Network is unreachable',)': /simple/wheel/
    Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f92a757b8>: Failed to establish a new connection: [Errno 101] Network is unreachable',)': /simple/wheel/
    Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f92a75208>: Failed to establish a new connection: [Errno 101] Network is unreachable',)': /simple/wheel/
    Could not find a version that satisfies the requirement wheel (from versions: )
  No matching distribution found for wheel

  ----------------------------------------
Command "/usr/bin/python3 -m pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-d3vvzxn3 --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple --trusted-host pypi.org --trusted-host pypi.python.org --trusted-host files.pythonhosted.org -- wheel setuptools Cython>=0.29.13 "numpy==1.13.3; python_version=='3.5' and platform_system!='AIX'" "numpy==1.13.3; python_version=='3.6' and platform_system!='AIX'" "numpy==1.14.5; python_version=='3.7' and platform_system!='AIX'" "numpy==1.17.3; python_version>='3.8' and platform_system!='AIX'" "numpy==1.16.0; python_version=='3.5' and platform_system=='AIX'" "numpy==1.16.0; python_version=='3.6' and platform_system=='AIX'" "numpy==1.16.0; python_version=='3.7' and platform_system=='AIX'" "numpy==1.17.3; python_version>='3.8' and platform_system=='AIX'" pybind11>=2.4.0" failed with error code 1 in None

[aschor]

Hi, python 3.10.4 here. Same issue, how can I overcome this ? :

pip install -v --proxy=http://127.0.0.1:3128 setuptools py7zr
Using pip 22.1.1 from C:\Applications\Anaconda3\envs\mpp2\lib\site-packages\pip (python 3.10)
Requirement already satisfied: setuptools in c:\applications\anaconda3\envs\mpp2\lib\site-packages (62.3.2)
Collecting py7zr
  Using cached py7zr-0.18.6.tar.gz (4.9 MB)
  Running command pip subprocess to install build dependencies
  WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', ConnectionResetError(10054, 'Une connexion existante a dû être fermée par l’hôte distant', None, 10054, None))': /simple/setuptools/
  WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', ConnectionResetError(10054, 'Une connexion existante a dû être fermée par l’hôte distant', None, 10054, None))': /simple/setuptools/
  WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', ConnectionResetError(10054, 'Une connexion existante a dû être fermée par l’hôte distant', None, 10054, None))': /simple/setuptools/
  WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', ConnectionResetError(10054, 'Une connexion existante a dû être fermée par l’hôte distant', None, 10054, None))': /simple/setuptools/
  WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', ConnectionResetError(10054, 'Une connexion existante a dû être fermée par l’hôte distant', None, 10054, None))': /simple/setuptools/
  ERROR: Could not find a version that satisfies the requirement setuptools>=58 (from versions: none)
  ERROR: No matching distribution found for setuptools>=58
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: 'C:\Applications\Anaconda3\envs\mpp2\python.exe' 'C:\Users\VDMC5301\AppData\Local\Temp\1\pip-standalone-pip-znf6sfv9\__env_pip__.zip\pip' install --ignore-installed --no-user --prefix 'C:\Users\VDMC5301\AppData\Local\Temp\1\pip-build-env-i2ltdhl0\overlay' --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=58' wheel 'setuptools_scm[toml]>=6.0.1'
  cwd: [inherit]
  Installing build dependencies ... error
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

OK, I found a trick to overcome the issue. Like explained in the docs, I created the directory and file %APPDATA%\pip\pip.ini and added this content (the proxy configuration). Then my package installs.

[global]
proxy = http://127.0.0.1:3128

[uranusjr]

More specifically, pip does not correctly propagate options to the subprocess when installing PEP 517 dependencies. Could you help test whether this behaviour still happens on the latest version?