Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

editable dependency causes wrong versions to be installed

See original GitHub issue

Issue description

An error is not reported at the end of a pipenv sync command if an installed dependency violates a constraint / does not match the version specified in the lockfile.

An error is not reported at the end of creating a lockfile via pipenv install if an installed dependency violates a constraint specified in the Pipenv file / newly created lockfile.

Looks like an editable dependency is key to causing this issue. May be related to https://github.com/pypa/pipenv/issues/3296

Note that there appears to be a datarace involved as SOMETIMES when I run pipenv install the versions that are installed match the generated/established lockfile and SOMETIMES they do not!

The test case I describe here will be able to repeat the issue consistently. It appears to happen whenever there is a dependency of an editable dependency that is loosely locked down ( > or >= or < or <= or * ) this rarely happens if at all when such a dependency is locked down exactly ( == )

Output from test:

STARTING TESTS


testing 'pipenv sync' operation; iteration = 1
Creating a virtualenv for this project…
Pipfile: /workspace/Pipfile
Using /usr/local/bin/python (3.7.5) to create virtualenv…
⠋ Creating virtual environment...Already using interpreter /usr/local/bin/python
Using base prefix '/usr/local'
New python executable in /workspace/.venv/bin/python
Installing setuptools, pip, wheel...
done.

✔ Successfully created virtual environment!
Virtualenv location: /workspace/.venv
Installing dependencies from Pipfile.lock (bfdd12)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 18/18 — 00:01:08
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
All dependencies are now up-to-date!

verifying...FAIL

the following python packages did not install correctly:
 amqp: expected 1.4.9 got 2.5.2 instead
 celery: expected 3.1.19 got 4.4.0 instead
 billiard: expected 3.3.0.23 got 3.6.1.0 instead

Expected result

I expect installed dependency versions to not violate constraints specified in the Pipenv file and I expect dependencies that are installed when running pipenv sync or pipenv install to match exactly the version specified in the lockfile - whether that lockfile be pre-existing or newly created.

If this does not happen, I expect an error to be reported.

( In addition, if the lockfile exists and I run pipenv sync I still expect the lockfile and Pipenv file to remain unchanged. Though there appears to be no issue around that combination of conditions I feel it should be enumerated. )

Actual result

Versions of dependencies that violate the top-most Pipenv file are installed when running pipenv install. Also they do not match the version stated in the newly created lockfile.

Versions of dependencies that do not match the lockfile are installed when running pipenv sync.

No error is thrown.

Steps to replicate

Made a couple of minor tests that express the problem: On a linux friendly system with make:

clone https://github.com/bottom-text-everywhere/test-python-project
cd test-python-project
install docker
make test

View the README.md file in the project root for more details.

There are two tests: calling pipenv sync and calling pipenv install which creates a lockfile.

$ pipenv --support

Pipenv version: '2018.11.26'

Pipenv location: '/usr/local/lib/python3.7/site-packages/pipenv'

Python location: '/usr/local/bin/python'

Python installations found:

3.7.5: /usr/local/bin/python
3.7.5: /usr/local/bin/python3.7m
3.7.3: /usr/bin/python3.7m
3.7.3: /usr/bin/python3.7
2.7.16: /usr/bin/python2

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.7.5',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '4.9.184-linuxkit',
 'platform_system': 'Linux',
 'platform_version': '#1 SMP Tue Jul 2 22:58:16 UTC 2019',
 'python_full_version': '3.7.5',
 'python_version': '3.7',
 'sys_platform': 'linux'}

System environment variables:

PIPENV_VENV_IN_PROJECT
HOSTNAME
PYTHON_VERSION
DOCKER_SHELL_ACTIVE
PWD
HOME
LANG
GPG_KEY
TERM
SHLVL
PYTHON_PIP_VERSION
PYTHON_GET_PIP_SHA256
PYTHON_GET_PIP_URL
PATH
_
PIP_DISABLE_PIP_VERSION_CHECK
PYTHONDONTWRITEBYTECODE
PIP_SHIMS_BASE_MODULE
PIP_PYTHON_PATH
PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

PIPENV_VENV_IN_PROJECT: 1

Debug–specific environment variables:

PATH: /usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
LANG: C.UTF-8
PWD: /workspace

Contents of Pipfile (‘/workspace/Pipfile’):

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
requests = "==2.22.0"
anyjson = "==0.3.3"
Babel = "==2.2.0"
certifi = "==2019.6.16"
chardet = "==3.0.4"
idna = "==2.8"
pytz = "==2015.7"
tornado = "==4.2"
urllib3 = "==1.25.3"
vine = "==1.3.0"
flower = {editable = true,git = "https://github.com/mher/flower",ref = "f20f43c858df3c0a94e5e10015fcd7d52089e3f0"}
celery = "==3.1.19"
kombu = "==3.0.37"
amqp = "==1.4.9"
billiard = "==3.3.0.23"
importlib-metadata = "==1.1.0"
zipp = "==0.6.0"
more-itertools = "==8.0.2"

[requires]
python_version = "3.7"

Contents of Pipfile.lock (‘/workspace/Pipfile.lock’):

{
    "_meta": {
        "hash": {
            "sha256": "5a05b2be1247f8f5df396dbe9a557d43c34516cb9b9c0ae3a3c2eb4b9fbfdd12"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.7"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "amqp": {
            "hashes": [
                "sha256:2dea4d16d073c902c3b89d9b96620fb6729ac0f7a923bbc777cb4ad827c0c61a",
                "sha256:e0ed0ce6b8ffe5690a2e856c7908dc557e0e605283d6885dd1361d79f2928908"
            ],
            "index": "pypi",
            "version": "==1.4.9"
        },
        "anyjson": {
            "hashes": [
                "sha256:37812d863c9ad3e35c0734c42e0bf0320ce8c3bed82cd20ad54cb34d158157ba"
            ],
            "index": "pypi",
            "version": "==0.3.3"
        },
        "babel": {
            "hashes": [
                "sha256:d8cb4c0e78148aee89560f9fe21587aa57739c975bb89ff66b1e842cc697428f",
                "sha256:fed07cbcdcb3de79b53a8220eebed21c93f8dbb3dbce1d9c6b1c4b09e8aecf2b"
            ],
            "index": "pypi",
            "version": "==2.2.0"
        },
        "billiard": {
            "hashes": [
                "sha256:204e75d390ef8f839c30a93b696bd842c3941916e15921745d05edc2a83868ab",
                "sha256:23cb71472712e96bff3e0d45763b7b8a99e5040385fffb96816028352c255682",
                "sha256:692a2a5a55ee39a42bcb7557930e2541da85df9ea81c6e24827f63b80cd39d0b",
                "sha256:82041dbaa62f7fde1464d7ab449978618a38b241b40c0d31dafabb36446635dc",
                "sha256:958fc9f8fd5cc9b936b2cb9d96f02aa5ec3613ba13ee7f089c77ff0bcc368fac",
                "sha256:c0cbe8d45ba8d8213ad68ef9a1881002a151569c9424d551634195a18c3a4160",
                "sha256:ccfe0419eb5e49f27ad35cf06e75360af903df6d576c66cb8073246d4e023e5c",
                "sha256:d4d2fed1a251ea58eed47b48db3778ebb92f5ff4407dc91869c6f41c3a9249d0"
            ],
            "index": "pypi",
            "version": "==3.3.0.23"
        },
        "celery": {
            "hashes": [
                "sha256:434b4e62308428b0e83fbcd1cebb8853beb230237b1b23484063fcf1ff610b6e",
                "sha256:bd6a21290c822ecf78947a7dbc8b2270d88a5fa08a7a2bfcd908bb3d5ed66b71"
            ],
            "index": "pypi",
            "version": "==3.1.19"
        },
        "certifi": {
            "hashes": [
                "sha256:046832c04d4e752f37383b628bc601a7ea7211496b4638f6514d0e5b9acc4939",
                "sha256:945e3ba63a0b9f577b1395204e13c3a231f9bc0223888be653286534e5873695"
            ],
            "index": "pypi",
            "version": "==2019.6.16"
        },
        "chardet": {
            "hashes": [
                "sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae",
                "sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691"
            ],
            "index": "pypi",
            "version": "==3.0.4"
        },
        "flower": {
            "editable": true,
            "git": "https://github.com/mher/flower",
            "ref": "f20f43c858df3c0a94e5e10015fcd7d52089e3f0"
        },
        "idna": {
            "hashes": [
                "sha256:c357b3f628cf53ae2c4c05627ecc484553142ca23264e593d327bcde5e9c3407",
                "sha256:ea8b7f6188e6fa117537c3df7da9fc686d485087abf6ac197f9c46432f7e4a3c"
            ],
            "index": "pypi",
            "version": "==2.8"
        },
        "importlib-metadata": {
            "hashes": [
                "sha256:b044f07694ef14a6683b097ba56bd081dbc7cdc7c7fe46011e499dfecc082f21",
                "sha256:e6ac600a142cf2db707b1998382cc7fc3b02befb7273876e01b8ad10b9652742"
            ],
            "index": "pypi",
            "version": "==1.1.0"
        },
        "kombu": {
            "hashes": [
                "sha256:7ceab743e3e974f3e5736082e8cc514c009e254e646d6167342e0e192aee81a6",
                "sha256:e064a00c66b4d1058cd2b0523fb8d98c82c18450244177b6c0f7913016642650"
            ],
            "index": "pypi",
            "version": "==3.0.37"
        },
        "more-itertools": {
            "hashes": [
                "sha256:b84b238cce0d9adad5ed87e745778d20a3f8487d0f0cb8b8a586816c7496458d",
                "sha256:c833ef592a0324bcc6a60e48440da07645063c453880c9477ceb22490aec1564"
            ],
            "index": "pypi",
            "version": "==8.0.2"
        },
        "pytz": {
            "hashes": [
                "sha256:3ede470d3d17ba3c07638dfa0d10452bc1b6e5ad326127a65ba77e6aaeb11bec",
                "sha256:8b6ce1c993909783bc96e0b4f34ea223bff7a4df2c90bdb9c4e0f1ac928689e3",
                "sha256:99266ef30a37e43932deec2b7ca73e83c8dbc3b9ff703ec73eca6b1dae6befea",
                "sha256:fbd26746772c24cb93c8b97cbdad5cb9e46c86bbdb1b9d8a743ee00e2fb1fc5d"
            ],
            "index": "pypi",
            "version": "==2015.7"
        },
        "requests": {
            "hashes": [
                "sha256:11e007a8a2aa0323f5a921e9e6a2d7e4e67d9877e85773fba9ba6419025cbeb4",
                "sha256:9cf5292fcd0f598c671cfc1e0d7d1a7f13bb8085e9a590f48c010551dc6c4b31"
            ],
            "index": "pypi",
            "version": "==2.22.0"
        },
        "tornado": {
            "hashes": [
                "sha256:e8b1207da67dbdceebfb291292b4ef1b547d6171525bec1b366853f923456a5f"
            ],
            "index": "pypi",
            "version": "==4.2"
        },
        "urllib3": {
            "hashes": [
                "sha256:b246607a25ac80bedac05c6f282e3cdaf3afb65420fd024ac94435cabe6e18d1",
                "sha256:dbe59173209418ae49d485b87d1681aefa36252ee85884c31346debd19463232"
            ],
            "index": "pypi",
            "version": "==1.25.3"
        },
        "vine": {
            "hashes": [
                "sha256:133ee6d7a9016f177ddeaf191c1f58421a1dcc6ee9a42c58b34bed40e1d2cd87",
                "sha256:ea4947cc56d1fd6f2095c8d543ee25dad966f78692528e68b4fada11ba3f98af"
            ],
            "index": "pypi",
            "version": "==1.3.0"
        },
        "zipp": {
            "hashes": [
                "sha256:3718b1cbcd963c7d4c5511a8240812904164b7f381b647143a89d3b98f9bcd8e",
                "sha256:f06903e9f1f43b12d371004b4ac7b06ab39a44adc747266928ae6debfa7b3335"
            ],
            "index": "pypi",
            "version": "==0.6.0"
        }
    },
    "develop": {}
}