Lock file differs depending on Python version of Pipenv installation

Pipenv seems to create a different lock file depending on whether Pipenv was installed with pip2 or pip3.

Example with pipenv installed using pip2 install --user pipenv:

$ which pipenv
/Users/martin/Library/Python/2.7/bin/pipenv

$ pipenv install apscheduler
Creating a virtualenv for this project…
⠋Using base prefix '/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6'
New python executable in /Users/martin/.local/share/virtualenvs/foo-mzLZXOua/bin/python3.6
Also creating executable in /Users/martin/.local/share/virtualenvs/foo-mzLZXOua/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /Users/martin/.local/share/virtualenvs/foo-mzLZXOua
Creating a Pipfile for this project…
Installing apscheduler…
Collecting apscheduler
  Using cached APScheduler-3.3.1-py2.py3-none-any.whl
Collecting tzlocal>=1.2 (from apscheduler)
Collecting pytz (from apscheduler)
  Using cached pytz-2017.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools>=0.7 in /Users/martin/.local/share/virtualenvs/foo-mzLZXOua/lib/python3.6/site-packages (from apscheduler)
Collecting six>=1.4.0 (from apscheduler)
  Using cached six-1.11.0-py2.py3-none-any.whl
Installing collected packages: pytz, tzlocal, six, apscheduler
Successfully installed apscheduler-3.3.1 pytz-2017.2 six-1.11.0 tzlocal-1.4

Adding apscheduler to Pipfile's [packages]…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (019ead)!

$ cat Pipfile
[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"

[dev-packages]

[packages]
apscheduler = "*"

$ cat Pipfile.lock
{
    "_meta": {
        "hash": {
            "sha256": "10b2336ecdb1ebf47f350b4b23ada696f0662969a53d4c798a51bf073c019ead"
        },
        "host-environment-markers": {
            "implementation_name": "cpython",
            "implementation_version": "3.6.3",
            "os_name": "posix",
            "platform_machine": "x86_64",
            "platform_python_implementation": "CPython",
            "platform_release": "17.0.0",
            "platform_system": "Darwin",
            "platform_version": "Darwin Kernel Version 17.0.0: Thu Aug 24 21:48:19 PDT 2017; root:xnu-4570.1.46~2/RELEASE_X86_64",
            "python_full_version": "3.6.3",
            "python_version": "3.6",
            "sys_platform": "darwin"
        },
        "pipfile-spec": 6,
        "requires": {},
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.python.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "apscheduler": {
            "hashes": [
                "sha256:bc9f96e498adb362beb5f1d3715a2570d100183add4ace5227c1a7d5dbaac900",
                "sha256:f68874dff1bdffcc6ce3adb7840c1e4d162c609a3e3f831351df30b75732767b"
            ],
            "version": "==3.3.1"
        },
        "funcsigs": {
            "hashes": [
                "sha256:330cc27ccbf7f1e992e69fef78261dc7c6569012cf397db8d3de0234e6c937ca",
                "sha256:a7bb0f2cf3a3fd1ab2732cb49eba4252c2af4240442415b4abce3b87022a8f50"
            ],
            "markers": "python_version == '2.7'",
            "version": "==1.0.2"
        },
        "futures": {
            "hashes": [
                "sha256:c4884a65654a7c45435063e14ae85280eb1f111d94e542396717ba9828c4337f",
                "sha256:51ecb45f0add83c806c68e4b06106f90db260585b25ef2abfcda0bd95c0132fd"
            ],
            "markers": "python_version == '2.7'",
            "version": "==3.1.1"
        },
        "pytz": {
            "hashes": [
                "sha256:c883c2d6670042c7bc1688645cac73dd2b03193d1f7a6847b6154e96890be06d",
                "sha256:03c9962afe00e503e2d96abab4e8998a0f84d4230fa57afe1e0528473698cdd9",
                "sha256:487e7d50710661116325747a9cd1744d3323f8e49748e287bc9e659060ec6bf9",
                "sha256:43f52d4c6a0be301d53ebd867de05e2926c35728b3260157d274635a0a947f1c",
                "sha256:d1d6729c85acea5423671382868627129432fba9a89ecbb248d8d1c7a9f01c67",
                "sha256:54a935085f7bf101f86b2aff75bd9672b435f51c3339db2ff616e66845f2b8f9",
                "sha256:39504670abb5dae77f56f8eb63823937ce727d7cdd0088e6909e6dcac0f89043",
                "sha256:ddc93b6d41cfb81266a27d23a79e13805d4a5521032b512643af8729041a81b4",
                "sha256:f5c056e8f62d45ba8215e5cb8f50dfccb198b4b9fbea8500674f3443e4689589"
            ],
            "version": "==2017.2"
        },
        "six": {
            "hashes": [
                "sha256:832dc0e10feb1aa2c68dcc57dbb658f1c7e65b9b61af69048abc87a2db00a0eb",
                "sha256:70e8a77beed4562e7f14fe23a786b54f6296e34344c23bc42f07b15018ff98e9"
            ],
            "version": "==1.11.0"
        },
        "tzlocal": {
            "hashes": [
                "sha256:05a2908f7fb1ba8843f03b2360d6ad314dbf2bce4644feb702ccd38527e13059"
            ],
            "version": "==1.4"
        }
    },
    "develop": {}
}

And pipenv installed with pip3 install --user pipenv:

$ which pipenv
/Users/martin/Library/Python/3.6/bin/pipenv

$ pipenv update
Updating all dependencies from Pipfile…
Found 3 installed package(s), purging…
Uninstalling APScheduler-3.3.1:
  Successfully uninstalled APScheduler-3.3.1
Uninstalling pytz-2017.2:
  Successfully uninstalled pytz-2017.2
Uninstalling tzlocal-1.4:
  Successfully uninstalled tzlocal-1.4

Environment now purged and fresh!
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (019ead)!
Installing dependencies from Pipfile.lock (019ead)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 4/4 — 00:00:01
To activate this project's virtualenv, run the following:
 $ pipenv shell
All dependencies are now up-to-date!

$ cat Pipfile.lock
{
    "_meta": {
        "hash": {
            "sha256": "10b2336ecdb1ebf47f350b4b23ada696f0662969a53d4c798a51bf073c019ead"
        },
        "host-environment-markers": {
            "implementation_name": "cpython",
            "implementation_version": "0",
            "os_name": "posix",
            "platform_machine": "x86_64",
            "platform_python_implementation": "CPython",
            "platform_release": "17.0.0",
            "platform_system": "Darwin",
            "platform_version": "Darwin Kernel Version 17.0.0: Thu Aug 24 21:48:19 PDT 2017; root:xnu-4570.1.46~2/RELEASE_X86_64",
            "python_full_version": "2.7.14",
            "python_version": "2.7",
            "sys_platform": "darwin"
        },
        "pipfile-spec": 6,
        "requires": {},
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.python.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "apscheduler": {
            "hashes": [
                "sha256:bc9f96e498adb362beb5f1d3715a2570d100183add4ace5227c1a7d5dbaac900",
                "sha256:f68874dff1bdffcc6ce3adb7840c1e4d162c609a3e3f831351df30b75732767b"
            ],
            "version": "==3.3.1"
        },
        "pytz": {
            "hashes": [
                "sha256:c883c2d6670042c7bc1688645cac73dd2b03193d1f7a6847b6154e96890be06d",
                "sha256:03c9962afe00e503e2d96abab4e8998a0f84d4230fa57afe1e0528473698cdd9",
                "sha256:487e7d50710661116325747a9cd1744d3323f8e49748e287bc9e659060ec6bf9",
                "sha256:43f52d4c6a0be301d53ebd867de05e2926c35728b3260157d274635a0a947f1c",
                "sha256:d1d6729c85acea5423671382868627129432fba9a89ecbb248d8d1c7a9f01c67",
                "sha256:54a935085f7bf101f86b2aff75bd9672b435f51c3339db2ff616e66845f2b8f9",
                "sha256:39504670abb5dae77f56f8eb63823937ce727d7cdd0088e6909e6dcac0f89043",
                "sha256:ddc93b6d41cfb81266a27d23a79e13805d4a5521032b512643af8729041a81b4",
                "sha256:f5c056e8f62d45ba8215e5cb8f50dfccb198b4b9fbea8500674f3443e4689589"
            ],
            "version": "==2017.2"
        },
        "six": {
            "hashes": [
                "sha256:832dc0e10feb1aa2c68dcc57dbb658f1c7e65b9b61af69048abc87a2db00a0eb",
                "sha256:70e8a77beed4562e7f14fe23a786b54f6296e34344c23bc42f07b15018ff98e9"
            ],
            "version": "==1.11.0"
        },
        "tzlocal": {
            "hashes": [
                "sha256:05a2908f7fb1ba8843f03b2360d6ad314dbf2bce4644feb702ccd38527e13059"
            ],
            "version": "==1.4"
        }
    },
    "develop": {}
}

Using pipenv --two update in the last example gives the same result.

This effectively means that it’s not possible to use a Python 3 Pipenv to generate a Pipfile.lock or requirements.txt that is usable in a Python 2 environment.

[ncoghlan]

As of v11, managing Py2 environments with Py3 (and vice-versa) should be markedly improved, as https://github.com/pypa/pipenv/commit/ecf78424c1740bae3511c731c032a3768154072c means the dependency resolution for lock file generation now runs in the context of the venv Python.

In particular, if a setuptools extension runs a subprocess (or otherwise bypasses pipenv’s attempt to override the nominal Python version), that subprocess will also run in the virtual environment, and hence see the correct version of Python (and other environmental details).

[ghost]

In our workflow, we use macOS for development & docker for deployment to Linux (AWS Lambda or EC2 instances). Our codebase is a mix of Python2 & Python 3 services deployed independently. While investigating Pipenv, I realized that the Pipfile.lock generated on macOS for a python2 project fails to work within docker. I have installed pipenv in python3 in macOS. The culprit is cryptography package which includes ipaddress packed on python < 3. The fix for this issue will provide the solution for this use case.

However, lockfile being dependant on the OS is contrary to my expectations set up by npm, go dep, nuget & other tools. If there are no plans to make lockfile independent of OS, this point should be clearly mentioned in the documentation.