pip install doesn't carry-on index-url certificate - failing build dependencies installation

Environment

• pip version: 20.2.3
• Python version: 3.6.8
• OS: Windows 10

Description

I’m using pip to install a package (iniconfig==1.1.0) from an internal secured repository, thus providing a certificate via command line args (--cert=...). The package is distributed as a source dist (not a wheel) and pip tries first to install the package build dependencies (Installing build dependencies ...). when it tries doing so, pip invokes a nested pip install command, carrying on the index-url but not the certificate, thus the installation fails. Note how in the sub pip install command the index is carried on but the certificate is not:

ERROR: Command errored out with exit status 1:
   command: 'bug_venv\Scripts\python.exe' 'bug_venv\lib\site-packages\pip' install --ignore-installed --no-user --prefix 'C:\Users\AWAIZM~1\AppData\Local\Temp\pip-build-env-cmo7_f0v\overlay' --no-warn-script-location --no-binary :none: --only-binary :none: -i https://il-app-61/repository/pypi_group/simple -- 'setuptools>=41.2.0' wheel 'setuptools_scm>3'

Expected behavior

certificate argument should be carried on to sub pip install commands

How to Reproduce

1. create a new venv with pip 20.2.3, setuptools 50.3.1 and wheel 0.35.1

python -m venv bug_venv
bug_venv\Scripts\python.exe -m pip install pip==20.2.3
bug_venv\Scripts\python.exe -m pip install -U setuptools==50.3.1 wheel==0.35.1

1. run pip install command with internal repo and certificate

bug_venv\Scripts\python.exe -m pip install --cert=<certificate file> --index-url=<https repo url> iniconfig==1.1.0 

1. install should fail with “certificate verify failed”

Output

bug_venv\Scripts\python -m pip install --cert=F:\views\g\qprism\QBuildSystemCore\python\build_infra\src\amat_build_system\resources\amat.pem.cer --index-url=https://il-app-61/repository/pypi_group/simple iniconfig==1.1.0
Looking in indexes: https://il-app-61/repository/pypi_group/simple
Collecting iniconfig==1.1.0
  Downloading https://il-app-61/repository/pypi_group/packages/iniconfig/1.1.0/iniconfig-1.1.0.tar.gz (8.1 kB)
  Installing build dependencies ... error
  ERROR: Command errored out with exit status 1:
   command: 'bug_venv\Scripts\python.exe' 'bug_venv\lib\site-packages\pip' install --ignore-installed --no-user --prefix 'C:\Users\AWAIZM~1\AppData\Local\Temp\pip-build-env-cmo7_f0v\overlay' --no-warn-script-location --no-binary :none: --only-binary :none: -i https://il-app-61/repository/pypi_group/simple -- 'setuptools>=41.2.0' wheel 'setuptools_scm>3'
       cwd: None
  Complete output (10 lines):
  Looking in indexes: https://il-app-61/repository/pypi_group/simple
  WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)': /repository/pypi_group/simple/setuptools/
  WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)': /repository/pypi_group/simple/setuptools/
  WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)': /repository/pypi_group/simple/setuptools/
  WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)': /repository/pypi_group/simple/setuptools/
  WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)': /repository/pypi_group/simple/setuptools/
  Could not fetch URL https://il-app-61/repository/pypi_group/simple/setuptools/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='il-app-61', port=443): Max retries exceeded with url: /repository/pypi_group/simple/setuptools/ (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)) - skipping
  ERROR: Could not find a version that satisfies the requirement setuptools>=41.2.0 (from versions: none)
  ERROR: No matching distribution found for setuptools>=41.2.0
  Could not fetch URL https://il-app-61/repository/pypi_group/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='il-app-61', port=443): Max retries exceeded with url: /repository/pypi_group/simple/pip/ (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)) - skipping
  ----------------------------------------
ERROR: Command errored out with exit status 1: 'bug_venv\Scripts\python.exe' 'bug_venv\lib\site-packages\pip' install --ignore-installed --no-user --prefix 'C:\Users\AWAIZM~1\AppData\Local\Temp\pip-build-env-cmo7_f0v\overlay' --no-warn-script-location --no-binary :none: --only-binary :none: -i https://il-app-61/repository/pypi_group/simple -- 'setuptools>=41.2.0' wheel 'setuptools_scm>3' Check the logs for full command output.

[awaizman1]

Hi @pfmoore @pradyunsg ,

Kindly would like to know if there is some fix planned for this or some workaround I can use.

Thanks

[pfmoore]

No fix currently planned/available, I’m afraid. As usual PRs would be welcome (but it’s not a simple fix!)

As a workaround, you could disable build isolation and manage the build environment manually.