Pipenv 2020.6.2 Falsly flags Homebrew's Python as a virtual environment

Hey Hey 👋,

First off. Thank you for all the work that has gone into the recent releases! I’m happy to use pipenv almost on a daily basis.

Issue description

It seems that the latest pipenv falsely flags my macOS + Python3 via Homebrew as a virtual environment.

The subsequent installation of dependencies does noting on existing projects or fails weirdly with python2 errors on new projects.

Expected result

$ pipenv install

Warning: the environment variable LANG is not set!
We recommend setting this in ~/.profile (or equivalent) for proper expected behavior.
Creating a virtualenv for this project…
Pipfile: /private/tmp/cool-project/Pipfile
Using /usr/local/bin/python3.7m (3.7.7) to create virtualenv…
⠴ Creating virtual environment...Using base prefix '/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7'
New python executable in /Users/nico/.local/share/virtualenvs/cool-project-gb-P45lz/bin/python3.7
Also creating executable in /Users/nico/.local/share/virtualenvs/cool-project-gb-P45lz/bin/python
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/local/bin/python3.7m

✔ Successfully created virtual environment!
Virtualenv location: /Users/nico/.local/share/virtualenvs/cool-project-gb-P45lz
Installing dependencies from Pipfile.lock (de0b8f)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 21/21 — 00:00:02
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.

Actual result on new projects

$ pipenv install

Courtesy Notice: Pipenv found itself running within a virtual environment, so it will automatically use that environment, instead of creating its own for any project. You can set PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and create its own instead. You can set PIPENV_VERBOSITY=-1 to suppress this warning.
Installing dependencies from Pipfile.lock (de0b8f)…
Failed to load paths: /bin/sh: /Users/nico/.local/share/virtualenvs/cool-project-gb-P45lz/bin/python: No such file or directory

Output:
Failed to load paths: /bin/sh: /Users/nico/.local/share/virtualenvs/cool-project-gb-P45lz/bin/python: No such file or directory

Output:
Failed to load paths: /bin/sh: /Users/nico/.local/share/virtualenvs/cool-project-gb-P45lz/bin/python: No such file or directory

Output:
An error occurred while installing cachetools==4.1.0 --hash=sha256:1d057645db16ca7fe1f3bd953558897603d6f0b9c51ed9d11eb4d071ec4e2aab --hash=sha256:de5d88f87781602201cde465d3afe837546663b168e8b39df67411b0bf10cefc! Will try again.
An error occurred while installing marshmallow==3.6.0 --hash=sha256:f88fe96434b1f0f476d54224d59333eba8ca1a203a2695683c1855675c4049a7 --hash=sha256:c2673233aa21dde264b84349dc2fd1dce5f30ed724a0a00e75426734de5b84ab! Will try again.
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 21/21 — 00:00:02
Installing initially failed dependencies…
[InstallError]:   File "/usr/local/lib/python3.7/site-packages/pipenv/cli/command.py", line 252, in install
[InstallError]:       site_packages=state.site_packages
[InstallError]:   File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 2062, in do_install
[InstallError]:       keep_outdated=keep_outdated
[InstallError]:   File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 1314, in do_init
[InstallError]:       pypi_mirror=pypi_mirror,
[InstallError]:   File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 901, in do_install_dependencies
[InstallError]:       retry_list, procs, failed_deps_queue, requirements_dir, **install_kwargs
[InstallError]:   File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 796, in batch_install
[InstallError]:       _cleanup_procs(procs, failed_deps_queue, retry=retry)
[InstallError]:   File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 703, in _cleanup_procs
[InstallError]:       raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
[pipenv.exceptions.InstallError]: ERROR: Could not find a version that satisfies the requirement cachetools==4.1.0 (from -r /var/folders/bm/nrklr36j2d92m7wz22_1nkym0000gn/T/pipenv-lc3jfmuu-requirements/pipenv-nnfotjey-requirement.txt (line 1)) (from versions: 0.0.0, 0.1.0, 0.2.0, 0.3.0, 0.3.1, 0.4.0, 0.5.0, 0.5.1, 0.6.0, 0.7.0, 0.7.1, 0.8.0, 0.8.1, 0.8.2, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.1.6, 2.0.0, 2.0.1, 2.1.0, 3.0.0, 3.1.0, 3.1.1)
[pipenv.exceptions.InstallError]: ERROR: No matching distribution found for cachetools==4.1.0 (from -r /var/folders/bm/nrklr36j2d92m7wz22_1nkym0000gn/T/pipenv-lc3jfmuu-requirements/pipenv-nnfotjey-requirement.txt (line 1))
ERROR: Couldn't install package: cachetools
 Package installation failed...
  ☤  ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 1/2 —

Actual result on existing projects

$ pipenv install

Courtesy Notice: Pipenv found itself running within a virtual environment, so it will automatically use that environment, instead of creating its own for any project. You can set PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and create its own instead. You can set PIPENV_VERBOSITY=-1 to suppress this warning.
Installing dependencies from Pipfile.lock (de0b8f)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/0 —

Steps to replicate

1. Use macOS (I’m on Mojave 10.14.6)
2. Install python using homebrew: brew install python a. Probably unrelated: I’ve also installed Python 3.8 using homebrew brew install python@3.8
3. Also make sure the install paths of homebrew are in your PATH. a. I think my setup is based upon this guide b. Make sure to reload/reboot any open sessions so the changes in PATH are reflected
4. Run pip3 install -U pipenv to install the latest pipenv
5. Run pipenv install in a python project
6. Acknowledge incorrect virtual environment detection because it spits out the “Pipenv found itself running within a virtual environment” message.

System info

$ pipenv --support

Pipenv version: '2020.6.2'

Pipenv location: '/usr/local/lib/python3.7/site-packages/pipenv'

Python location: '/usr/local/opt/python/bin/python3.7'

Python installations found:

• 3.7.7: /usr/local/bin/python3
• 3.7.7: /usr/local/bin/python3.7m
• 3.7.7: /usr/local/bin/python3.7
• 3.6.7: /usr/local/bin/python3.6
• 3.6.7: /usr/local/bin/python3.6m
• 2.7.17: /usr/local/bin/python2
• 2.7.17: /usr/local/bin/python2.7
• 2.7.16: /usr/bin/python2.7

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.7.7',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '18.7.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 18.7.0: Mon Apr 27 20:09:39 PDT '
                     '2020; root:xnu-4903.278.35~1/RELEASE_X86_64',
 'python_full_version': '3.7.7',
 'python_version': '3.7',
 'sys_platform': 'darwin'}

System environment variables:

• TERM_SESSION_ID
• SSH_AUTH_SOCK
• LC_TERMINAL_VERSION
• Apple_PubSub_Socket_Render
• COLORFGBG
• ITERM_PROFILE
• XPC_FLAGS
• PWD
• SHELL
• LC_CTYPE
• TERM_PROGRAM_VERSION
• TERM_PROGRAM
• PATH
• LC_TERMINAL
• COLORTERM
• TERM
• HOME
• TMPDIR
• USER
• XPC_SERVICE_NAME
• LOGNAME
• ITERM_SESSION_ID
• __CF_USER_TEXT_ENCODING
• SHLVL
• OLDPWD
• ZSH
• PAGER
• LESS
• LSCOLORS
• ZNT_REPO_DIR
• ZNT_CONFIG_DIR
• NVM_DIR
• GOPATH
• _
• PIP_DISABLE_PIP_VERSION_CHECK
• PYTHONDONTWRITEBYTECODE
• PIP_SHIMS_BASE_MODULE
• PIP_PYTHON_PATH
• PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

• PATH: /usr/local/opt/openjdk@11/bin:/Users/nico/Library/Android/sdk/platform-tools:/Users/nico/Library/Python/3.7/bin:/Users/nico/Projects/go/bin:/usr/local/bin:/usr/local/sbin:/Users/nico/.poetry/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/go/bin:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/usr/local/CrossPack-AVR/bin:/Users/nico/.composer/vendor/bin
• SHELL: /bin/zsh
• PWD: /tmp/cool-project

---

Contents of Pipfile (‘/private/tmp/cool-project/Pipfile’):

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
flask = "*"
docker = "*"
waitress = "*"
flask-basicauth = "*"
sentry-sdk = {extras = ["flask"],version = "*"}
flask-marshmallow = "*"
cachetools = "*"

[requires]
python_version = "3.7"

Contents of Pipfile.lock (‘/private/tmp/cool-project/Pipfile.lock’):

{
    "_meta": {
        "hash": {
            "sha256": "09c8342a4c3edee1ba3cffcf27e6d97613ed3c020e1c1c77b62a9fd7ffde0b8f"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.7"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "blinker": {
            "hashes": [
                "sha256:471aee25f3992bd325afa3772f1063dbdbbca947a041b8b89466dc00d606f8b6"
            ],
            "version": "==1.4"
        },
        "cachetools": {
            "hashes": [
                "sha256:1d057645db16ca7fe1f3bd953558897603d6f0b9c51ed9d11eb4d071ec4e2aab",
                "sha256:de5d88f87781602201cde465d3afe837546663b168e8b39df67411b0bf10cefc"
            ],
            "index": "pypi",
            "version": "==4.1.0"
        },
        "certifi": {
            "hashes": [
                "sha256:1d987a998c75633c40847cc966fcf5904906c920a7f17ef374f5aa4282abd304",
                "sha256:51fcb31174be6e6664c5f69e3e1691a2d72a1a12e90f872cbdb1567eb47b6519"
            ],
            "version": "==2020.4.5.1"
        },
        "chardet": {
            "hashes": [
                "sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae",
                "sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691"
            ],
            "version": "==3.0.4"
        },
        "click": {
            "hashes": [
                "sha256:d2b5255c7c6349bc1bd1e59e08cd12acbbd63ce649f2588755783aa94dfb6b1a",
                "sha256:dacca89f4bfadd5de3d7489b7c8a566eee0d3676333fbb50030263894c38c0dc"
            ],
            "version": "==7.1.2"
        },
        "docker": {
            "hashes": [
                "sha256:1c2ddb7a047b2599d1faec00889561316c674f7099427b9c51e8cb804114b553",
                "sha256:ddae66620ab5f4bce769f64bcd7934f880c8abe6aa50986298db56735d0f722e"
            ],
            "index": "pypi",
            "version": "==4.2.0"
        },
        "flask": {
            "hashes": [
                "sha256:4efa1ae2d7c9865af48986de8aeb8504bf32c7f3d6fdc9353d34b21f4b127060",
                "sha256:8a4fdd8936eba2512e9c85df320a37e694c93945b33ef33c89946a340a238557"
            ],
            "index": "pypi",
            "version": "==1.1.2"
        },
        "flask-basicauth": {
            "hashes": [
                "sha256:df5ebd489dc0914c224419da059d991eb72988a01cdd4b956d52932ce7d501ff"
            ],
            "index": "pypi",
            "version": "==0.2.0"
        },
        "flask-marshmallow": {
            "hashes": [
                "sha256:6e6aec171b8e092e0eafaf035ff5b8637bf3a58ab46f568c4c1bab02f2a3c196",
                "sha256:a1685536e7ab5abdc712bbc1ac1a6b0b50951a368502f7985e7d1c27b3c21e59"
            ],
            "index": "pypi",
            "version": "==0.12.0"
        },
        "idna": {
            "hashes": [
                "sha256:7588d1c14ae4c77d74036e8c22ff447b26d0fde8f007354fd48a7814db15b7cb",
                "sha256:a068a21ceac8a4d63dbfd964670474107f541babbd2250d61922f029858365fa"
            ],
            "version": "==2.9"
        },
        "itsdangerous": {
            "hashes": [
                "sha256:321b033d07f2a4136d3ec762eac9f16a10ccd60f53c0c91af90217ace7ba1f19",
                "sha256:b12271b2047cb23eeb98c8b5622e2e5c5e9abd9784a153e9d8ef9cb4dd09d749"
            ],
            "version": "==1.1.0"
        },
        "jinja2": {
            "hashes": [
                "sha256:89aab215427ef59c34ad58735269eb58b1a5808103067f7bb9d5836c651b3bb0",
                "sha256:f0a4641d3cf955324a89c04f3d94663aa4d638abe8f733ecd3582848e1c37035"
            ],
            "version": "==2.11.2"
        },
        "markupsafe": {
            "hashes": [
                "sha256:00bc623926325b26bb9605ae9eae8a215691f33cae5df11ca5424f06f2d1f473",
                "sha256:09027a7803a62ca78792ad89403b1b7a73a01c8cb65909cd876f7fcebd79b161",
                "sha256:09c4b7f37d6c648cb13f9230d847adf22f8171b1ccc4d5682398e77f40309235",
                "sha256:1027c282dad077d0bae18be6794e6b6b8c91d58ed8a8d89a89d59693b9131db5",
                "sha256:13d3144e1e340870b25e7b10b98d779608c02016d5184cfb9927a9f10c689f42",
                "sha256:24982cc2533820871eba85ba648cd53d8623687ff11cbb805be4ff7b4c971aff",
                "sha256:29872e92839765e546828bb7754a68c418d927cd064fd4708fab9fe9c8bb116b",
                "sha256:43a55c2930bbc139570ac2452adf3d70cdbb3cfe5912c71cdce1c2c6bbd9c5d1",
                "sha256:46c99d2de99945ec5cb54f23c8cd5689f6d7177305ebff350a58ce5f8de1669e",
                "sha256:500d4957e52ddc3351cabf489e79c91c17f6e0899158447047588650b5e69183",
                "sha256:535f6fc4d397c1563d08b88e485c3496cf5784e927af890fb3c3aac7f933ec66",
                "sha256:596510de112c685489095da617b5bcbbac7dd6384aeebeda4df6025d0256a81b",
                "sha256:62fe6c95e3ec8a7fad637b7f3d372c15ec1caa01ab47926cfdf7a75b40e0eac1",
                "sha256:6788b695d50a51edb699cb55e35487e430fa21f1ed838122d722e0ff0ac5ba15",
                "sha256:6dd73240d2af64df90aa7c4e7481e23825ea70af4b4922f8ede5b9e35f78a3b1",
                "sha256:717ba8fe3ae9cc0006d7c451f0bb265ee07739daf76355d06366154ee68d221e",
                "sha256:79855e1c5b8da654cf486b830bd42c06e8780cea587384cf6545b7d9ac013a0b",
                "sha256:7c1699dfe0cf8ff607dbdcc1e9b9af1755371f92a68f706051cc8c37d447c905",
                "sha256:88e5fcfb52ee7b911e8bb6d6aa2fd21fbecc674eadd44118a9cc3863f938e735",
                "sha256:8defac2f2ccd6805ebf65f5eeb132adcf2ab57aa11fdf4c0dd5169a004710e7d",
                "sha256:98c7086708b163d425c67c7a91bad6e466bb99d797aa64f965e9d25c12111a5e",
                "sha256:9add70b36c5666a2ed02b43b335fe19002ee5235efd4b8a89bfcf9005bebac0d",
                "sha256:9bf40443012702a1d2070043cb6291650a0841ece432556f784f004937f0f32c",
                "sha256:ade5e387d2ad0d7ebf59146cc00c8044acbd863725f887353a10df825fc8ae21",
                "sha256:b00c1de48212e4cc9603895652c5c410df699856a2853135b3967591e4beebc2",
                "sha256:b1282f8c00509d99fef04d8ba936b156d419be841854fe901d8ae224c59f0be5",
                "sha256:b2051432115498d3562c084a49bba65d97cf251f5a331c64a12ee7e04dacc51b",
                "sha256:ba59edeaa2fc6114428f1637ffff42da1e311e29382d81b339c1817d37ec93c6",
                "sha256:c8716a48d94b06bb3b2524c2b77e055fb313aeb4ea620c8dd03a105574ba704f",
                "sha256:cd5df75523866410809ca100dc9681e301e3c27567cf498077e8551b6d20e42f",
                "sha256:cdb132fc825c38e1aeec2c8aa9338310d29d337bebbd7baa06889d09a60a1fa2",
                "sha256:e249096428b3ae81b08327a63a485ad0878de3fb939049038579ac0ef61e17e7",
                "sha256:e8313f01ba26fbbe36c7be1966a7b7424942f670f38e666995b88d012765b9be"
            ],
            "version": "==1.1.1"
        },
        "marshmallow": {
            "hashes": [
                "sha256:c2673233aa21dde264b84349dc2fd1dce5f30ed724a0a00e75426734de5b84ab",
                "sha256:f88fe96434b1f0f476d54224d59333eba8ca1a203a2695683c1855675c4049a7"
            ],
            "version": "==3.6.0"
        },
        "requests": {
            "hashes": [
                "sha256:43999036bfa82904b6af1d99e4882b560e5e2c68e5c4b0aa03b655f3d7d73fee",
                "sha256:b3f43d496c6daba4493e7c431722aeb7dbc6288f52a6e04e7b6023b0247817e6"
            ],
            "version": "==2.23.0"
        },
        "sentry-sdk": {
            "extras": [
                "flask"
            ],
            "hashes": [
                "sha256:0e5e947d0f7a969314aa23669a94a9712be5a688ff069ff7b9fc36c66adc160c",
                "sha256:799a8bf76b012e3030a881be00e97bc0b922ce35dde699c6537122b751d80e2c"
            ],
            "index": "pypi",
            "version": "==0.14.4"
        },
        "six": {
            "hashes": [
                "sha256:236bdbdce46e6e6a3d61a337c0f8b763ca1e8717c03b369e87a7ec7ce1319c0a",
                "sha256:8f3cd2e254d8f793e7f3d6d9df77b92252b52637291d0f0da013c76ea2724b6c"
            ],
            "version": "==1.14.0"
        },
        "urllib3": {
            "hashes": [
                "sha256:3018294ebefce6572a474f0604c2021e33b3fd8006ecd11d62107a5d2a963527",
                "sha256:88206b0eb87e6d677d424843ac5209e3fb9d0190d0ee169599165ec25e9d9115"
            ],
            "version": "==1.25.9"
        },
        "waitress": {
            "hashes": [
                "sha256:045b3efc3d97c93362173ab1dfc159b52cfa22b46c3334ffc805dbdbf0e4309e",
                "sha256:77ff3f3226931a1d7d8624c5371de07c8e90c7e5d80c5cc660d72659aaf23f38"
            ],
            "index": "pypi",
            "version": "==1.4.3"
        },
        "websocket-client": {
            "hashes": [
                "sha256:0fc45c961324d79c781bab301359d5a1b00b13ad1b10415a4780229ef71a5549",
                "sha256:d735b91d6d1692a6a181f2a8c9e0238e5f6373356f561bb9dc4c7af36f452010"
            ],
            "version": "==0.57.0"
        },
        "werkzeug": {
            "hashes": [
                "sha256:2de2a5db0baeae7b2d2664949077c2ac63fbd16d98da0ff71837f7d1dea3fd43",
                "sha256:6c80b1e5ad3665290ea39320b91e1be1e0d5f60652b964a3070216de83d2e47c"
            ],
            "version": "==1.0.1"
        }
    },
    "develop": {}
}

Notes

Assuming that a Homebrew installed python is not supposed to be a virtual environment, I tried digging into this myself and found a possible cause of the bug. Please note that i’m making some assumptions so I’d love to hear your input on this 😃

The last release introduced an _OSX_VENV variable that is later-on used in the method is_in_virtualenv() to determine if the command is running inside a virtual environment. This variable stores the removed value of an __PYVENV_LAUNCHER__ environment variable, which is originally implemented to prevent faulty shebangs.

I suspect that using the OSX_VENV variable is too aggressive in determining if we are running inside a virtual environment. I’ve checked on a different mac setup with a Python 3.7.2 and it seems that the __PYVENV_LAUNCHER__ is also set there.

My suggested fix would be to remove the _OSX_VENV variable, this works on my machine. Although I’m not sure if this reintroduces https://github.com/pypa/pipenv/pull/4284

Edit: Formatting and typo’s

[chrish42]

So, I’m trying to be as kind as possible. I know how much of the opensource community (to which I am also a part of) is powered by volunteers. But pipenv has been broken for at least a good portion of the macOS Python user base for almost two months now. I know there’s are tradeoffs to be made in selecting a threshold for when to do bugfix point release vs when to wait for the next normal release. I would like to very respectfully suggest that this project has its threshold for what requires a bugfix point release set too high currently. In the meantime, I can’t use or recommend a core tool that stays broken for this long for a significant portion of its userbase.

(If the project changes its mind and wishes to do a bugfix point release, and is lacking the time to port over the patch, I’d be more than happy to help.)

[jpz]

I just thumbed up the issue a month or so ago, but didn’t post a comment here. Yet the problem here just had me determine pipenv to be non-viable to use.

I just have a very standard MacOS set up I have (i.e. latest Python 3.8 installed with homebrew).

My feedback here is meant with goodwill and respect to the people that put their time in to support the project, and just want to give the feedback that the issue here impacts engagement with the project for another individual here, also.