Pipenv lock doesn't take Pipfile into account when processing package sub-dependencies
See original GitHub issueIssue description
After the release of pylint 2, I discovered this issue trying to install plugins such as pylint-quotes and pytest-pylint:
Pipenv seems to ignore dependency versions in Pipfile when looking through sub-dependencies (dependencies of packages specified in Pipfile).
Expected result
Pipenv should account for Pipfile when resolving sub-depencendies to avoid reporting erroneous conflicts
Actual result
Pipenv incorrectly reports dependency conflicts when none actually exist.
Steps to replicate
Example Pipfile:
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[packages]
pylint = "==1.9"
pylint-quotes = "==0.1.9"
$ pipenv lock -v
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
using sources: [{'url': 'https://pypi.org/simple', 'verify_ssl': True, 'name': 'pypi'}]
Using pip: -i https://pypi.org/simple
ROUND 1
Current constraints:
pylint==1.9 (from -r /var/folders/rj/210lgzrd4q5ccsyby0rdxg0924842d/T/pipenv-vkxnjh1h-requirements/pipenv-2cbbt1ga-constraints.txt (line 2))
pylint-quotes==0.1.9 (from -r /var/folders/rj/210lgzrd4q5ccsyby0rdxg0924842d/T/pipenv-vkxnjh1h-requirements/pipenv-2cbbt1ga-constraints.txt (line 3))
Finding the best candidates:
found candidate pylint==1.9 (constraint was ==1.9)
found candidate pylint-quotes==0.1.9 (constraint was ==0.1.9)
Finding secondary dependencies:
pylint-quotes==0.1.9 requires astroid>=2.0.1, isort>=4.2.5, lazy-object-proxy, mccabe, pylint-quotes==0.1.9, pylint>=1.7.6, six, wrapt
pylint==1.9 requires astroid<2.0,>=1.6, isort>=4.2.5, lazy-object-proxy, mccabe, pylint==1.9, six, wrapt
New dependencies found in this round:
adding ['astroid', '<2.0,>=1.6,>=2.0.1', '[]']
adding ['isort', '>=4.2.5', '[]']
adding ['lazy-object-proxy', '', '[]']
adding ['mccabe', '', '[]']
adding ['pylint', '==1.9,>=1.7.6', '[]']
adding ['pylint-quotes', '==0.1.9', '[]']
adding ['six', '', '[]']
adding ['wrapt', '', '[]']
Removed dependencies in this round:
Unsafe dependencies in this round:
------------------------------------------------------------
Result of round 1: not stable
ROUND 2
Current constraints:
astroid<2.0,>=1.6,>=2.0.1
isort>=4.2.5
lazy-object-proxy
mccabe
pylint==1.9,>=1.7.6 (from -r /var/folders/rj/210lgzrd4q5ccsyby0rdxg0924842d/T/pipenv-vkxnjh1h-requirements/pipenv-2cbbt1ga-constraints.txt (line 2))
pylint-quotes==0.1.9 (from -r /var/folders/rj/210lgzrd4q5ccsyby0rdxg0924842d/T/pipenv-vkxnjh1h-requirements/pipenv-2cbbt1ga-constraints.txt (line 3))
six
wrapt
Finding the best candidates:
Using pip: -i https://pypi.org/simple
ROUND 1
Current constraints:
pylint==1.9 (from -r /var/folders/rj/210lgzrd4q5ccsyby0rdxg0924842d/T/pipenv-vkxnjh1h-requirements/pipenv-eulvhsyk-constraints.txt (line 2))
pylint-quotes==0.1.9 (from -r /var/folders/rj/210lgzrd4q5ccsyby0rdxg0924842d/T/pipenv-vkxnjh1h-requirements/pipenv-eulvhsyk-constraints.txt (line 3))
Finding the best candidates:
found candidate pylint==1.9 (constraint was ==1.9)
found candidate pylint-quotes==0.1.9 (constraint was ==0.1.9)
Finding secondary dependencies:
pylint-quotes==0.1.9 requires astroid>=2.0.1, isort>=4.2.5, lazy-object-proxy, mccabe, pylint-quotes==0.1.9, pylint>=1.7.6, six, wrapt
pylint==1.9 requires astroid<2.0,>=1.6, isort>=4.2.5, lazy-object-proxy, mccabe, pylint==1.9, six, wrapt
New dependencies found in this round:
adding ['astroid', '<2.0,>=1.6,>=2.0.1', '[]']
adding ['isort', '>=4.2.5', '[]']
adding ['lazy-object-proxy', '', '[]']
adding ['mccabe', '', '[]']
adding ['pylint', '==1.9,>=1.7.6', '[]']
adding ['pylint-quotes', '==0.1.9', '[]']
adding ['six', '', '[]']
adding ['wrapt', '', '[]']
Removed dependencies in this round:
Unsafe dependencies in this round:
------------------------------------------------------------
Result of round 1: not stable
ROUND 2
Current constraints:
astroid<2.0,>=1.6,>=2.0.1
isort>=4.2.5
lazy-object-proxy
mccabe
pylint==1.9,>=1.7.6 (from -r /var/folders/rj/210lgzrd4q5ccsyby0rdxg0924842d/T/pipenv-vkxnjh1h-requirements/pipenv-eulvhsyk-constraints.txt (line 2))
pylint-quotes==0.1.9 (from -r /var/folders/rj/210lgzrd4q5ccsyby0rdxg0924842d/T/pipenv-vkxnjh1h-requirements/pipenv-eulvhsyk-constraints.txt (line 3))
six
wrapt
Finding the best candidates:
Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
Hint: try $ pipenv lock --pre if it is a pre-release dependency.
Could not find a version that matches astroid<2.0,>=1.6,>=2.0.1
Tried: 1.0.0, 1.0.1, 1.1.0, 1.1.1, 1.2.0, 1.2.1, 1.3.0, 1.3.0, 1.3.1, 1.3.1, 1.3.2, 1.3.2, 1.3.3, 1.3.3, 1.3.4, 1.3.4, 1.3.5, 1.3.5, 1.3.6, 1.3.6, 1.3.7, 1.3.7, 1.3.8, 1.3.8, 1.4.0, 1.4.0, 1.4.1, 1.4.1, 1.4.2, 1.4.2, 1.4.3, 1.4.3, 1.4.4, 1.4.4, 1.4.5, 1.4.5, 1.4.6, 1.4.6, 1.4.7, 1.4.7, 1.4.8, 1.4.8, 1.4.9, 1.4.9, 1.5.0, 1.5.0, 1.5.1, 1.5.1, 1.5.2, 1.5.2, 1.5.3, 1.5.3, 1.6.0, 1.6.0, 1.6.1, 1.6.1, 1.6.2, 1.6.2, 1.6.3, 1.6.3, 1.6.4, 1.6.4, 1.6.5, 1.6.5, 2.0, 2.0, 2.0.1, 2.0.1
Skipped pre-versions: 2.0.0.dev0, 2.0.0.dev0, 2.0.0.dev1, 2.0.0.dev1, 2.0.0.dev2, 2.0.0.dev2, 2.0.0.dev3, 2.0.0.dev3, 2.0.0.dev4, 2.0.0.dev4
There are incompatible versions in the resolved dependencies.
Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
Hint: try $ pipenv lock --pre if it is a pre-release dependency.
Could not find a version that matches astroid<2.0,>=1.6,>=2.0.1
Tried: 1.0.0, 1.0.1, 1.1.0, 1.1.1, 1.2.0, 1.2.1, 1.3.0, 1.3.0, 1.3.1, 1.3.1, 1.3.2, 1.3.2, 1.3.3, 1.3.3, 1.3.4, 1.3.4, 1.3.5, 1.3.5, 1.3.6, 1.3.6, 1.3.7, 1.3.7, 1.3.8, 1.3.8, 1.4.0, 1.4.0, 1.4.1, 1.4.1, 1.4.2, 1.4.2, 1.4.3, 1.4.3, 1.4.4, 1.4.4, 1.4.5, 1.4.5, 1.4.6, 1.4.6, 1.4.7, 1.4.7, 1.4.8, 1.4.8, 1.4.9, 1.4.9, 1.5.0, 1.5.0, 1.5.1, 1.5.1, 1.5.2, 1.5.2, 1.5.3, 1.5.3, 1.6.0, 1.6.0, 1.6.1, 1.6.1, 1.6.2, 1.6.2, 1.6.3, 1.6.3, 1.6.4, 1.6.4, 1.6.5, 1.6.5, 2.0, 2.0, 2.0.1, 2.0.1
Skipped pre-versions: 2.0.0.dev0, 2.0.0.dev0, 2.0.0.dev1, 2.0.0.dev1, 2.0.0.dev2, 2.0.0.dev2, 2.0.0.dev3, 2.0.0.dev3, 2.0.0.dev4, 2.0.0.dev4
There are incompatible versions in the resolved dependencies.
pylint 1.9 requires astroid<2.0,>=1.6
pylint-quotes 0.1.9’s only dependency is pylint>=1.7.6
which can be seen in its setup.cfg (or by running pip show after installing it):
...
install_requires=[
'pylint>=1.7.6',
],
...
However, Pipenv incorrectly reports that pylint-quotes requires astroid>=2.0.1
.
Further investigation revealed that there is one related package that does require astroid>=2.0.1
, pylint==2.0.1
which we are not installing. It looks to me like Pipenv is seeing the pylint requirement of pylint-quotes and then checking the dependencies of the latest version of pylint rather than the one actually specified in the pipfile.
$ pipenv --support
Pipenv version: '2018.7.1'
Pipenv location: '/usr/local/Cellar/pipenv/2018.7.1/libexec/lib/python3.7/site-packages/pipenv'
Python location: '/usr/local/Cellar/pipenv/2018.7.1/libexec/bin/python3.7'
Other Python installations in PATH
:
-
2.7
:/usr/local/bin/python2.7
-
2.7
:/usr/local/bin/python2.7
-
2.7
:/Users/<user>/.pyenv/shims/python2.7
-
2.7
:/usr/local/bin/python2.7
-
2.7
:/usr/bin/python2.7
-
3.6
:/Users/<user>/.pyenv/shims/python3.6m
-
3.6
:/Users/<user>/.pyenv/shims/python3.6
-
3.7
:/Users/<user>/.pyenv/shims/python3.7
-
2.7.15
:/usr/local/bin/python
-
3.6.5
:/Users/<user>/.pyenv/shims/python
-
2.7.15
:/usr/local/bin/python
-
2.7.10
:/usr/bin/python
-
2.7.15
:/usr/local/bin/python2
-
2.7.15
:/Users/<user>/.pyenv/shims/python2
-
2.7.15
:/usr/local/bin/python2
-
3.6.5
:/Users/<user>/.pyenv/shims/python3
PEP 508 Information:
{'implementation_name': 'cpython',
'implementation_version': '3.7.0',
'os_name': 'posix',
'platform_machine': 'x86_64',
'platform_python_implementation': 'CPython',
'platform_release': '17.7.0',
'platform_system': 'Darwin',
'platform_version': 'Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT '
'2018; root:xnu-4570.71.2~1/RELEASE_X86_64',
'python_full_version': '3.7.0',
'python_version': '3.7',
'sys_platform': 'darwin'}
System environment variables:
PATH
LDFLAGS
MANPATH
_fzf_orig_completion_tee
TERM_PROGRAM
_fzf_orig_completion_find
_fzf_orig_completion_diff
_fzf_orig_completion_javac
PYENV_ROOT
TERM
SHELL
_fzf_orig_completion_curl
CPPFLAGS
_fzf_orig_completion_mv
TMPDIR
_fzf_orig_completion_patch
Apple_PubSub_Socket_Render
_fzf_orig_completion_perl
TERM_PROGRAM_VERSION
_fzf_orig_completion_python
_fzf_orig_completion_du
_fzf_orig_completion_bunzip2
TERM_SESSION_ID
_fzf_orig_completion_less
_fzf_orig_completion_rmdir
_fzf_orig_completion_tail
_fzf_orig_completion_head
PYENV_VERSION
_fzf_orig_completion_jar
_fzf_orig_completion_svn
_fzf_orig_completion_telnet
USER
_fzf_orig_completion_g__
_fzf_orig_completion_wc
_fzf_orig_completion_ftp
_fzf_orig_completion_gzip
PYENV_DIR
SSH_AUTH_SOCK
_fzf_orig_completion_view
_fzf_orig_completion_export
__CF_USER_TEXT_ENCODING
PYENV_VIRTUALENV_INIT
_fzf_orig_completion_grep
_fzf_orig_completion_gvim
_fzf_orig_completion_java
_fzf_orig_completion_unzip
_fzf_orig_completion_sftp
PWD
_fzf_orig_completion_rm
_fzf_orig_completion_ls
_fzf_orig_completion_uniq
EDITOR
_fzf_orig_completion_cat
_fzf_orig_completion_chown
_fzf_orig_completion_bzip2
LANG
ITERM_PROFILE
PYENV_HOOK_PATH
XPC_FLAGS
_fzf_orig_completion_cd
_fzf_orig_completion_vi
_fzf_orig_completion_tar
XPC_SERVICE_NAME
_fzf_orig_completion_kill
PYENV_SHELL
SHLVL
COLORFGBG
HOME
ITERM_SESSION_ID
LOGNAME
_fzf_orig_completion_vim
VISUAL
_fzf_orig_completion_awk
_fzf_orig_completion_ld
_fzf_orig_completion_sort
_fzf_orig_completion_ssh
_fzf_orig_completion_gunzip
PKG_CONFIG_PATH
GOPATH
_fzf_orig_completion_rsync
_fzf_orig_completion_gcc
_fzf_orig_completion_emacs
DISPLAY
_fzf_orig_completion_cp
_fzf_orig_completion_scp
_fzf_orig_completion_ln
_fzf_orig_completion_sed
_fzf_orig_completion_git
COLORTERM
PYTHONDONTWRITEBYTECODE
PIP_PYTHON_PATH
Pipenv–specific environment variables:
Debug–specific environment variables:
PATH
:/usr/local/Cellar/pipenv/2018.7.1/libexec/tools:/usr/local/bin:/Users/<user>/.pyenv/libexec:/Users/<user>/.pyenv/plugins/python-build/bin:/Users/<user>/.pyenv/plugins/pyenv-virtualenv/bin:/Users/<user>/.pyenv/plugins/pyenv-update/bin:/Users/<user>/.pyenv/plugins/pyenv-installer/bin:/Users/<user>/.pyenv/plugins/pyenv-doctor/bin:/Users/<user>/.local/bin:/usr/local/opt/coreutils/libexec/gnubin:/usr/local/opt/libarchive/bin:/usr/local/sbin:/usr/local/opt/go/libexec/bin:/Users/<user>/go/bin:/Users/<user>/.pyenv/plugins/pyenv-virtualenv/shims:/Users/<user>/.pyenv/shims:/Users/<user>/.pyenv/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/VMware Fusion.app/Contents/Public:/usr/local/MacGPG2/bin:/opt/X11/bin:/Users/<user>/.vim/plugged/fzf/bin
SHELL
:/usr/local/bin/bash
EDITOR
:vim
LANG
:en_US.UTF-8
PWD
:/Users/<user>/Documents/Source/pipenvdeps
Contents of Pipfile
(‘/Users/<user>/Documents/Source/pipenvdeps/Pipfile’):
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[packages]
pylint = "==1.9"
pylint-quotes = "==0.1.9"
[dev-packages]
[requires]
python_version = "3.7"
Contents of Pipfile.lock
(‘/Users/<user>/Documents/Source/pipenvdeps/Pipfile.lock’):
{
"_meta": {
"hash": {
"sha256": "8401a941de091e385a3679312b16f3d165c68c9f8d79c6963b1d3e3a741dbe53"
},
"pipfile-spec": 6,
"requires": {
"python_version": "3.7"
},
"sources": [
{
"name": "pypi",
"url": "https://pypi.org/simple",
"verify_ssl": true
}
]
},
"default": {
"astroid": {
"hashes": [
"sha256:0ef2bf9f07c3150929b25e8e61b5198c27b0dca195e156f0e4d5bdd89185ca1a",
"sha256:fc9b582dba0366e63540982c3944a9230cbc6f303641c51483fa547dcc22393a"
],
"version": "==1.6.5"
},
"isort": {
"hashes": [
"sha256:1153601da39a25b14ddc54955dbbacbb6b2d19135386699e2ad58517953b34af",
"sha256:b9c40e9750f3d77e6e4d441d8b0266cf555e7cdabdcff33c4fd06366ca761ef8",
"sha256:ec9ef8f4a9bc6f71eec99e1806bfa2de401650d996c59330782b89a5555c1497"
],
"markers": "python_version != '3.3.*' and python_version >= '2.7' and python_version != '3.1.*' and python_version != '3.0.*' and python_version != '3.2.*'",
"version": "==4.3.4"
},
"lazy-object-proxy": {
"hashes": [
"sha256:0ce34342b419bd8f018e6666bfef729aec3edf62345a53b537a4dcc115746a33",
"sha256:1b668120716eb7ee21d8a38815e5eb3bb8211117d9a90b0f8e21722c0758cc39",
"sha256:209615b0fe4624d79e50220ce3310ca1a9445fd8e6d3572a896e7f9146bbf019",
"sha256:27bf62cb2b1a2068d443ff7097ee33393f8483b570b475db8ebf7e1cba64f088",
"sha256:27ea6fd1c02dcc78172a82fc37fcc0992a94e4cecf53cb6d73f11749825bd98b",
"sha256:2c1b21b44ac9beb0fc848d3993924147ba45c4ebc24be19825e57aabbe74a99e",
"sha256:2df72ab12046a3496a92476020a1a0abf78b2a7db9ff4dc2036b8dd980203ae6",
"sha256:320ffd3de9699d3892048baee45ebfbbf9388a7d65d832d7e580243ade426d2b",
"sha256:50e3b9a464d5d08cc5227413db0d1c4707b6172e4d4d915c1c70e4de0bbff1f5",
"sha256:5276db7ff62bb7b52f77f1f51ed58850e315154249aceb42e7f4c611f0f847ff",
"sha256:61a6cf00dcb1a7f0c773ed4acc509cb636af2d6337a08f362413c76b2b47a8dd",
"sha256:6ae6c4cb59f199d8827c5a07546b2ab7e85d262acaccaacd49b62f53f7c456f7",
"sha256:7661d401d60d8bf15bb5da39e4dd72f5d764c5aff5a86ef52a042506e3e970ff",
"sha256:7bd527f36a605c914efca5d3d014170b2cb184723e423d26b1fb2fd9108e264d",
"sha256:7cb54db3535c8686ea12e9535eb087d32421184eacc6939ef15ef50f83a5e7e2",
"sha256:7f3a2d740291f7f2c111d86a1c4851b70fb000a6c8883a59660d95ad57b9df35",
"sha256:81304b7d8e9c824d058087dcb89144842c8e0dea6d281c031f59f0acf66963d4",
"sha256:933947e8b4fbe617a51528b09851685138b49d511af0b6c0da2539115d6d4514",
"sha256:94223d7f060301b3a8c09c9b3bc3294b56b2188e7d8179c762a1cda72c979252",
"sha256:ab3ca49afcb47058393b0122428358d2fbe0408cf99f1b58b295cfeb4ed39109",
"sha256:bd6292f565ca46dee4e737ebcc20742e3b5be2b01556dafe169f6c65d088875f",
"sha256:cb924aa3e4a3fb644d0c463cad5bc2572649a6a3f68a7f8e4fbe44aaa6d77e4c",
"sha256:d0fc7a286feac9077ec52a927fc9fe8fe2fabab95426722be4c953c9a8bede92",
"sha256:ddc34786490a6e4ec0a855d401034cbd1242ef186c20d79d2166d6a4bd449577",
"sha256:e34b155e36fa9da7e1b7c738ed7767fc9491a62ec6af70fe9da4a057759edc2d",
"sha256:e5b9e8f6bda48460b7b143c3821b21b452cb3a835e6bbd5dd33aa0c8d3f5137d",
"sha256:e81ebf6c5ee9684be8f2c87563880f93eedd56dd2b6146d8a725b50b7e5adb0f",
"sha256:eb91be369f945f10d3a49f5f9be8b3d0b93a4c2be8f8a5b83b0571b8123e0a7a",
"sha256:f460d1ceb0e4a5dcb2a652db0904224f367c9b3c1470d5a7683c0480e582468b"
],
"version": "==1.3.1"
},
"mccabe": {
"hashes": [
"sha256:ab8a6258860da4b6677da4bd2fe5dc2c659cff31b3ee4f7f5d64e79735b80d42",
"sha256:dd8d182285a0fe56bace7f45b5e7d1a6ebcbf524e8f3bd87eb0f125271b8831f"
],
"version": "==0.6.1"
},
"pylint": {
"hashes": [
"sha256:b719c86a7395ea0c0ec8030c2a7a2b4fad573ee50460f9948fabb1811d72094f",
"sha256:cf1be367296e9e534a5cb420186ce99f63f17c2b855fcb4321a3e20ce51502cd"
],
"index": "pypi",
"version": "==1.9"
},
"six": {
"hashes": [
"sha256:70e8a77beed4562e7f14fe23a786b54f6296e34344c23bc42f07b15018ff98e9",
"sha256:832dc0e10feb1aa2c68dcc57dbb658f1c7e65b9b61af69048abc87a2db00a0eb"
],
"version": "==1.11.0"
},
"wrapt": {
"hashes": [
"sha256:d4d560d479f2c21e1b5443bbd15fe7ec4b37fe7e53d335d3b9b0a7b1226fe3c6"
],
"version": "==1.10.11"
}
},
"develop": {}
}
Issue Analytics
- State:
- Created 5 years ago
- Comments:12 (4 by maintainers)
You might have luck if you don’t have seperate dev dependencies
On Sat, Sep 22, 2018, 23:52 Bob Spryn notifications@github.com wrote:
The implementation still has a lot of things going on right now, and there isn’t a very convenient way to make it work with Pipenv yet, unfortunately. If you’re interested, however, I would very much like people to throw real-world examples at it and see what happens. But be aware: this is definitely not ready for production.
Here’s the implementation: https://github.com/sarugaku/resolvelib
Setup would be something like:
Use this command to test the resolver out:
This will emit a ton of output to show what happens in the resolver, and a final
STABLE PINS
section that includes what is actually resolved.There are some known issues about dependencies that use
file
,path
,git
etc. to specify the package. This is already being worked on.