Provide index to pip for installation of sdist build dependencies

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Windows 10, Version 1909
• Poetry version: 1.1.3
• Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/ajacquah/4b6cab23073fb21c03eaa436b911f89d

Issue

Pip as invoked by poetry installs build depdencies from pypi.org instead of custom repository. And since I am running this behind a firewall with no proxy settings this fails. My understanding was that poetry would use default repository for all package actions. Can you please take a look?

To reproduce:

1. Create new project. $poetry new --src test-pkg
2. Add a private default source. This source is a mirror of pypi.org + internal packages.
3. From limited connectivity environment, try to add black dev dependency: poetry add black -D

Result stack trace:

Command C:\Source\test-pkg\.venv\Scripts\pip.exe install --no-deps file:///C:/Users/ajacquah/AppData/Local/pypoetry/Cache/artifacts/8b/b9/b7/d4ec53ca89b288b6be53361d96da13d397c1dfaf70aa7b48b127e6c727/black-20.8b1.tar.gz errored with the following return code 1, and output:
  Processing c:\users\ajacquah\appdata\local\pypoetry\cache\artifacts\8b\b9\b7\d4ec53ca89b288b6be53361d96da13d397c1dfaf70aa7b48b127e6c727\black-20.8b1.tar.gz
    Installing build dependencies: started
    Installing build dependencies: still running...
    Installing build dependencies: still running...
    Installing build dependencies: still running...
    Installing build dependencies: still running...
    Installing build dependencies: still running...
    Installing build dependencies: still running...
    Installing build dependencies: finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: 'C:\Source\rails-test\.venv\Scripts\python.exe' 'C:\Source\rails-test\.venv\lib\site-packages\pip' install --ignore-installed --no-user --prefix 'C:\Users\ajacquah\AppData\Local\Temp\pip-build-env-x0saifh5\overlay' --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=41.0' setuptools-scm wheel
         cwd: None
    Complete output (7 lines):
    WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0x00000278EC4CB670>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/setuptools/
    WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0x00000278EC4CBAF0>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/setuptools/
    WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0x00000278EC4CBC40>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/setuptools/
    WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0x00000278EC4CBEE0>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/setuptools/
    WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0x00000278EC4A3BB0>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/setuptools/
    ERROR: Could not find a version that satisfies the requirement setuptools>=41.0 (from versions: none)
    ERROR: No matching distribution found for setuptools>=41.0
    ----------------------------------------
  ERROR: Command errored out with exit status 1: 'C:\Source\test-pkg\.venv\Scripts\python.exe' 'C:\Source\test-pkg\.venv\lib\site-packages\pip' install --ignore-installed --no-user --prefix 'C:\Users\ajacquah\AppData\Local\Temp\pip-build-env-x0saifh5\overlay' --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=41.0' setuptools-scm wheel Check the logs for full command output.


  at c:\python38\lib\site-packages\poetry\utils\env.py:948 in _run
       944│                 output = subprocess.check_output(
       945│                     cmd, stderr=subprocess.STDOUT, **kwargs
       946│                 )
       947│         except CalledProcessError as e:
    →  948│             raise EnvCommandError(e, input=input_)
       949│
       950│         return decode(output)
       951│
       952│     def execute(self, bin, *args, **kwargs):

Thanks for your help!

[aja-intel]

hello @abn and maintainers. Any further thoughts on this? I’ve seen a few suggestion for work-arounds on related issues but it would be great to see this working for the configured default package index.

[neersighted]

As you can see, this has been triaged. Nobody has worked on implementing anything for five reasons:

1. it’s a rather niche issue with lots of sharp edges and implementing index-passing to pip is going to be a bit tricky to get the semantics right on
2. we’re in the middle of a refactor of source handling and will have to figure out how to map the new semantics to pip
3. it’s unclear if this is even correct as Poetry supports source priorities and pip does not – unexpected packages may be pulled in
4. passing authentication information to pip is also fraught, both code-wise and implementation-wise
5. we’re working on a new installer instead of using pip, that will let Poetry provide the build-system.requires directly and sidestep this entirely

If you need to set PIP_INDEX_URL, you should set in /etc/pip.conf, which represents system-wide configuration and usually will not include any options that can cause problems for Poetry (though of course, you can specify any option there; please keep it minimal!).