Pipfile.lock doesn't preserve some `extras` information

I also filed a possibly related issue here https://github.com/kennethreitz/pipenv/issues/1013

Environment

1. Linux
2. pipenv is running on python 3.4.3
3. Pipenv version = 8.3.1

Expected result

Should be able to reinstall extra dependencies from a Pipfile.lock

In this example raven[flask] in python 2 depends on Blinker and Flask because of the flask extra, and depends on contextlib2 because its python < 3.4; so these dependencies should be installed.

https://github.com/getsentry/raven-python/blob/master/setup.py

Actual result

pipenv install raven[flask] works as expected but after recreating the virtual environment, installing from Pipfile.lock doesn’t install all extra dependencies and some extra dependencies are absent from the Pipfile.lock.

Steps to replicate

Showing pipenv install raven[flask] installs all expected packages

chris $ pipenv --python 2.7
Virtualenv already exists!
Removing existing virtualenv…
Creating a virtualenv for this project…
Using /my/version/of/python2.7 to create virtualenv…
⠋Running virtualenv with interpreter /my/version/of/python2.7
New python executable in /home/chris/.local/share/virtualenvs/demos-8_UOIBMV/bin/python2.7
Also creating executable in /home/chris/.local/share/virtualenvs/demos-8_UOIBMV/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /home/chris/.local/share/virtualenvs/demos-8_UOIBMV
Creating a Pipfile for this project…
chris $ pipenv install raven[flask]
Installing raven[flask]…
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒Collecting raven[flask]
  Using cached raven-6.3.0-py2.py3-none-any.whl
Collecting contextlib2; python_version < "3.2" (from raven[flask])
  Using cached contextlib2-0.5.5-py2.py3-none-any.whl
Collecting blinker>=1.1; extra == "flask" (from raven[flask])
Collecting Flask>=0.8; extra == "flask" (from raven[flask])
  Using cached Flask-0.12.2-py2.py3-none-any.whl
Collecting itsdangerous>=0.21 (from Flask>=0.8; extra == "flask"->raven[flask])
Collecting Jinja2>=2.4 (from Flask>=0.8; extra == "flask"->raven[flask])
  Using cached Jinja2-2.9.6-py2.py3-none-any.whl
Collecting Werkzeug>=0.7 (from Flask>=0.8; extra == "flask"->raven[flask])
  Using cached Werkzeug-0.12.2-py2.py3-none-any.whl
Collecting click>=2.0 (from Flask>=0.8; extra == "flask"->raven[flask])
  Using cached click-6.7-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23 (from Jinja2>=2.4->Flask>=0.8; extra == "flask"->raven[flask])
Installing collected packages: contextlib2, blinker, itsdangerous, MarkupSafe, Jinja2, Werkzeug, click, Flask, raven
Successfully installed Flask-0.12.2 Jinja2-2.9.6 MarkupSafe-1.0 Werkzeug-0.12.2 blinker-1.4 click-6.7 contextlib2-0.5.5 itsdangerous-0.24 raven-6.3.0

Adding raven[flask] to Pipfile's [packages]…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (812aac)!
chris $ pipenv graph
blinker==1.4
Flask==0.12.2
  - click [required: >=2.0, installed: 6.7]
  - itsdangerous [required: >=0.21, installed: 0.24]
  - Jinja2 [required: >=2.4, installed: 2.9.6]
    - MarkupSafe [required: >=0.23, installed: 1.0]
  - Werkzeug [required: >=0.7, installed: 0.12.2]
raven==6.3.0
  - contextlib2 [required: Any, installed: 0.5.5]

chris $

Showing pipenv install doesn’t install some extra dependencies

chris $ pipenv --python 2.7
Virtualenv already exists!
Removing existing virtualenv…
Creating a virtualenv for this project…
Using /my/version/of/python2.7 to create virtualenv…
⠋  Running virtualenv with interpreter /my/version/of/python2.7
New python executable in /home/chris/.local/share/virtualenvs/demos-8_UOIBMV/bin/python2.7
Also creating executable in /home/chris/.local/share/virtualenvs/demos-8_UOIBMV/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /home/chris/.local/share/virtualenvs/demos-8_UOIBMV
chris $ pipenv install
Installing dependencies from Pipfile.lock (812aac)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 8/8 — 00:00:02
To activate this project's virtualenv, run the following:
 $ pipenv shell
chris $ pipenv graph
blinker==1.4
Flask==0.12.2
  - click [required: >=2.0, installed: 6.7]
  - itsdangerous [required: >=0.21, installed: 0.24]
  - Jinja2 [required: >=2.4, installed: 2.9.6]
    - MarkupSafe [required: >=0.23, installed: 1.0]
  - Werkzeug [required: >=0.7, installed: 0.12.2]
raven==6.3.0
  - contextlib2 [required: Any, installed: ?]

chris $

Showing pipenv install --skip-lock installs expected dependencies but pipinstall upgrade uninstalls some.

chris $ pipenv --python 2.7
Virtualenv already exists!
Removing existing virtualenv…
Creating a virtualenv for this project…
Using /my/version/of/python2.7 to create virtualenv…
⠋  Running virtualenv with interpreter /my/version/of/python2.7
New python executable in /home/chris/.local/share/virtualenvs/demos-8_UOIBMV/bin/python2.7
Also creating executable in /home/chris/.local/share/virtualenvs/demos-8_UOIBMV/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /home/chris/.local/share/virtualenvs/demos-8_UOIBMV
chris $ pipenv install --skip-lock
Installing dependencies from Pipfile…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 1/1 — 00:00:05
To activate this project's virtualenv, run the following:
 $ pipenv shell
chris $ pipenv graph
blinker==1.4
Flask==0.12.2
  - click [required: >=2.0, installed: 6.7]
  - itsdangerous [required: >=0.21, installed: 0.24]
  - Jinja2 [required: >=2.4, installed: 2.9.6]
    - MarkupSafe [required: >=0.23, installed: 1.0]
  - Werkzeug [required: >=0.7, installed: 0.12.2]
raven==6.3.0
  - contextlib2 [required: Any, installed: 0.5.5]

chris $ pipenv update
Updating all dependencies from Pipfile…
Found 9 installed package(s), purging…
Uninstalling blinker-1.4:
  Successfully uninstalled blinker-1.4
Uninstalling click-6.7:
  Successfully uninstalled click-6.7
Uninstalling contextlib2-0.5.5:
  Successfully uninstalled contextlib2-0.5.5
Uninstalling Flask-0.12.2:
  Successfully uninstalled Flask-0.12.2
Uninstalling itsdangerous-0.24:
  Successfully uninstalled itsdangerous-0.24
Uninstalling Jinja2-2.9.6:
  Successfully uninstalled Jinja2-2.9.6
Uninstalling MarkupSafe-1.0:
  Successfully uninstalled MarkupSafe-1.0
Uninstalling raven-6.3.0:
  Successfully uninstalled raven-6.3.0
Uninstalling Werkzeug-0.12.2:
  Successfully uninstalled Werkzeug-0.12.2

Environment now purged and fresh!
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (812aac)!
Installing dependencies from Pipfile.lock (812aac)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 8/8 — 00:00:02
To activate this project's virtualenv, run the following:
 $ pipenv shell
All dependencies are now up-to-date!
chris $ pipenv graph
blinker==1.4
Flask==0.12.2
  - click [required: >=2.0, installed: 6.7]
  - itsdangerous [required: >=0.21, installed: 0.24]
  - Jinja2 [required: >=2.4, installed: 2.9.6]
    - MarkupSafe [required: >=0.23, installed: 1.0]
  - Werkzeug [required: >=0.7, installed: 0.12.2]
raven==6.3.0
  - contextlib2 [required: Any, installed: ?]

chris $

Pipfile

[[source]]

name = "pypi"
url = "https://pypi.python.org/simple"
verify_ssl = true


[dev-packages]



[packages]

raven = {extras = ["flask"]}


[requires]

python_version = "2.7"

Pipfile.lock

{
    "_meta": {
        "hash": {
            "sha256": "79f6cd7950e9c9c7dfc51d1b941d4faa0a554593b2af880134c66358f5812aac"
        },
        "host-environment-markers": "REDACTED",
        "pipfile-spec": 6,
        "requires": {
            "python_version": "2.7"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.python.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "blinker": {
            "hashes": [
                "sha256:471aee25f3992bd325afa3772f1063dbdbbca947a041b8b89466dc00d606f8b6"
            ],
            "version": "==1.4"
        },
        "click": {
            "hashes": [
                "sha256:29f99fc6125fbc931b758dc053b3114e55c77a6e4c6c3a2674a2dc986016381d",
                "sha256:f15516df478d5a56180fbf80e68f206010e6d160fc39fa508b65e035fd75130b"
            ],
            "version": "==6.7"
        },
        "flask": {
            "hashes": [
                "sha256:0749df235e3ff61ac108f69ac178c9770caeaccad2509cb762ce1f65570a8856",
                "sha256:49f44461237b69ecd901cc7ce66feea0319b9158743dd27a2899962ab214dac1"
            ],
            "version": "==0.12.2"
        },
        "itsdangerous": {
            "hashes": [
                "sha256:cbb3fcf8d3e33df861709ecaf89d9e6629cff0a217bc2848f1b41cd30d360519"
            ],
            "version": "==0.24"
        },
        "jinja2": {
            "hashes": [
                "sha256:2231bace0dfd8d2bf1e5d7e41239c06c9e0ded46e70cc1094a0aa64b0afeb054",
                "sha256:ddaa01a212cd6d641401cb01b605f4a4d9f37bfc93043d7f760ec70fb99ff9ff"
            ],
            "version": "==2.9.6"
        },
        "markupsafe": {
            "hashes": [
                "sha256:a6be69091dac236ea9c6bc7d012beab42010fa914c459791d627dad4910eb665"
            ],
            "version": "==1.0"
        },
        "raven": {
            "hashes": [
                "sha256:cb644fb12ee886a341041dcd533540dfc82619a50bf0b7c587af070054bd2c7f",
                "sha256:f3e465a545dcdb6a387d1fcb199d08f786ba3732d7ce6aa681718b04da6aedf1"
            ],
            "version": "==6.3.0"
        },
        "werkzeug": {
            "hashes": [
                "sha256:e8549c143af3ce6559699a01e26fa4174f4c591dbee0a499f3cd4c3781cdec3d",
                "sha256:903a7b87b74635244548b30d30db4c8947fe64c5198f58899ddcd3a13c23bb26"
            ],
            "version": "==0.12.2"
        }
    },
    "develop": {}
}

[techalchemy]

Nailed it! Seen this one enough by now to know right away what’s up there

[sethwoodworth]

@techalchemy Yes! I have precisely that. A folder named airflow and a folder named alembic as the case might be. I can move the folders back to their current values when PR #958 is merged.