Public dependencies are not resolved properly on private packages
See original GitHub issueWhen a public package is listed as a dependency in a private package, the package will fail to install because it doesn’t exist in the private index.
For example, if you specify an additional source in your Pipfile
:
[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"
[[source]]
url = "http://localhost:8080/"
name = "local"
This is the following output from running pipenv install --verbose local_package
Installing local_package…
⠋Installing u'local_package'
$ "/Users/guido/.local/share/virtualenvs/src-GqcYiTnw/bin/pip" install --verbose "local_package" -i https://pypi.python.org/simple --exists-action w
⠏$ "/Users/guido/.local/share/virtualenvs/src-GqcYiTnw/bin/pip" install --verbose "local_package" -i http://localhost:8080/ --exists-action w
Collecting local_package
1 location(s) to search for versions of local-package:
* http://localhost:8080/local-package/
Getting page http://localhost:8080/local-package/
Starting new HTTP connection (1): localhost
"GET /local-package/ HTTP/1.1" 200 None
Analyzing links from page http://localhost:8080/local-package/
Found link http://localhost:8080/local-package/local_package-0.1.1-py3-none-any.whl#md5=ce978ae83f1fec0526e7f34c45d27810 (from http://localhost:8080/local-package/), version: 0.1.1
Using version 0.1.1 (newest of versions: 0.1.1)
"GET /local-package/local_package-0.1.1-py3-none-any.whl HTTP/1.1" 200 7273
Downloading http://localhost:8080/local-package/local_package-0.1.1-py3-none-any.whl
Downloading from URL http://localhost:8080/local-package/local_package-0.1.1-py3-none-any.whl#md5=ce978ae83f1fec0526e7f34c45d27810 (from http://localhost:8080/local-package/)
Collecting boto3 (from local_package)
1 location(s) to search for versions of boto3:
* http://localhost:8080/boto3/
Getting page http://localhost:8080/boto3/
"GET /boto3/ HTTP/1.1" 403 None
Could not fetch URL http://localhost:8080/boto3/: 403 Client Error: Forbidden for url: http://localhost:8080/boto3/ - skipping
Cleaning up...
Exception information:
Traceback (most recent call last):
File "/Users/guido/.local/share/virtualenvs/src-GqcYiTnw/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/Users/guido/.local/share/virtualenvs/src-GqcYiTnw/lib/python3.6/site-packages/pip/commands/install.py", line 335, in run
wb.build(autobuilding=True)
File "/Users/guido/.local/share/virtualenvs/src-GqcYiTnw/lib/python3.6/site-packages/pip/wheel.py", line 749, in build
self.requirement_set.prepare_files(self.finder)
File "/Users/guido/.local/share/virtualenvs/src-GqcYiTnw/lib/python3.6/site-packages/pip/req/req_set.py", line 380, in prepare_files
ignore_dependencies=self.ignore_dependencies))
File "/Users/guido/.local/share/virtualenvs/src-GqcYiTnw/lib/python3.6/site-packages/pip/req/req_set.py", line 554, in _prepare_file
require_hashes
File "/Users/guido/.local/share/virtualenvs/src-GqcYiTnw/lib/python3.6/site-packages/pip/req/req_install.py", line 278, in populate_link
self.link = finder.find_requirement(self, upgrade)
File "/Users/guido/.local/share/virtualenvs/src-GqcYiTnw/lib/python3.6/site-packages/pip/index.py", line 514, in find_requirement
'No matching distribution found for %s' % req
pip.exceptions.DistributionNotFound: No matching distribution found for boto3 (from local_package)
Error: An error occurred while installing local_package!
Could not find a version that satisfies the requirement boto3 (from local_package) (from versions: )
No matching distribution found for boto3 (from local_package)
It seems that the issue is with pipenv using -i
instead of --extra-index-url
If I run pip install --extra-index-url http://localhost:8080 local_package
everything is installed correctly.
Specifying the index
for local_package
under [packages]
doesn’t help.
Describe your environment
- OS Type: macOS Sierra
- Python version: Python 3.6.3
- Pipenv version: pipenv, version 10.1.2
Expected result
I would expect for the public package (in the case above, boto3
), to be fetched from the public source by default.
Actual result
The public package is fetched from the private source.
Steps to replicate
Set up a private pypi index, add source to Pipfile, and run install command from above.
Issue Analytics
- State:
- Created 6 years ago
- Reactions:4
- Comments:26 (18 by maintainers)
Hey @kennethreitz, I don’t think d5b9b36 has quite solved this issue. I’ve attempted to reproduce by publishing a package to Test PyPI that references a package on regular PyPI.
Expected behaviour
Installs
pipenv-test-private-package
from TestPyPI and discovers dependency onpipenv-test-public-package
which is available on regular PyPI.Works when running pip directly:
Actual behaviour
Fails with the following error message
Steps to reproduce
Using
pipenv
version1.15.2
, runpipenv install
using the followingPipfile
For simplicity’s sake, I’ve hosted the contents of the private and public packages at the following gists:
My
python -m pipenv.help
is:Pipenv version:
'11.5.2'
Pipenv location:
'/usr/local/lib/python2.7/site-packages/pipenv'
Python location:
'/usr/local/opt/python/bin/python2.7'
Other Python installations in
PATH
:2.6
:/usr/bin/python2.6
2.6
:/usr/bin/python2.6
2.7
:/usr/local/bin/python2.7
2.7
:/usr/local/bin/python2.7
2.7
:/usr/bin/python2.7
3.6
:/usr/local/bin/python3.6m
3.6
:/usr/local/bin/python3.6
2.7.13
:/usr/local/bin/python
2.7.10
:/usr/bin/python
2.7.13
:/usr/local/bin/python2
3.6.4
:/usr/local/bin/python3
PEP 508 Information:
System environment variables:
TERM_PROGRAM_VERSION
LOGNAME
USER
NVM_DIR
HOME
PATH
PYTHONUNBUFFERED
DISPLAY
TERM_PROGRAM
LANG
TERM
Apple_PubSub_Socket_Render
COLORFGBG
SHLVL
SECURITYSESSIONID
XPC_FLAGS
ITERM_SESSION_ID
EDITOR
TERM_SESSION_ID
XPC_SERVICE_NAME
SSH_AUTH_SOCK
SHELL
ITERM_PROFILE
PIP_PYTHON_PATH
TMPDIR
__CF_USER_TEXT_ENCODING
PWD
COLORTERM
I think we understand the issue. Here is my proposed solution, if there are any concerns raise them here and we can discuss:
Our current implementation iterates over the indexes and attempts to install each top level package using each index. It used to fail silently, obviously that doesn’t work anymore. This is problematic for a couple of reasons
My proposal is this:
EXTRA_INDEX_URL
FIND_LINKS
location rather than anEXTRA_INDEX_URL
which would enable local wheel repositories and the likeThoughts?