Virtualenv needs to be re-created for minor Pyhton updates on Homebrew (a.k.a please use venv!)

Issue description

Python versions 3.3+ bundle a venv module for creating virtualenvs (python -m venv). The virtualenvs created by venv are tidier and supported by Python upstream itself.

There is also a major issue with virtualenv on macOS using Homebrew: it creates symlinks specific to the currently installed package version. This means every time Homebrew updates Python (including minor version updates), I need to recreate all virtualenvs. This gets annoying quickly. venv virtualenvs, however, are not affected by this issue.

Expected result

Virtualenv created with venv:

% python3 -m venv ./venv
% . ./venv/bin/activate
(venv) % tree -L 4 $VIRTUAL_ENV
/private/tmp/envtest/venv
├── bin
│   ├── activate
│   ├── activate.csh
│   ├── activate.fish
│   ├── easy_install
│   ├── easy_install-3.7
│   ├── pip
│   ├── pip3
│   ├── pip3.7
│   ├── python -> python3
│   └── python3 -> /usr/local/bin/python3
├── include
├── lib
│   └── python3.7
│       └── site-packages
│           ├── __pycache__
│           ├── easy_install.py
│           ├── pip
│           ├── pip-19.0.3.dist-info
│           ├── pkg_resources
│           ├── setuptools
│           └── setuptools-40.8.0.dist-info
└── pyvenv.cfg

11 directories, 12 files

Actual result + Steps to replicate

 % cd /tmp/envtest
 % pipenv shell
Creating a virtualenv for this project…
Pipfile: /private/tmp/envtest/Pipfile
Using /usr/local/opt/python/bin/python3.7 (3.7.4) to create virtualenv…
⠼ Creating virtual environment...Already using interpreter /usr/local/opt/python/bin/python3.7
Using base prefix '/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7'
New python executable in /Users/marti.raudsepp/.local/share/virtualenvs/envtest-pJJWjZNx/bin/python3.7
Also creating executable in /Users/marti.raudsepp/.local/share/virtualenvs/envtest-pJJWjZNx/bin/python
Installing setuptools, pip, wheel...
done.

✔ Successfully created virtual environment!
Virtualenv location: /Users/marti.raudsepp/.local/share/virtualenvs/envtest-pJJWjZNx
Creating a Pipfile for this project…
Launching subshell in virtual environment…
 . /Users/marti.raudsepp/.local/share/virtualenvs/envtest-pJJWjZNx/bin/activate
%  . /Users/marti.raudsepp/.local/share/virtualenvs/envtest-pJJWjZNx/bin/activate
(envtest) % tree $VIRTUAL_ENV -L 3
/Users/$USER/.local/share/virtualenvs/envtest-pJJWjZNx
├── bin
│   ├── activate
│   ├── activate.csh
│   ├── activate.fish
│   ├── activate.ps1
│   ├── activate.xsh
│   ├── activate_this.py
│   ├── easy_install
│   ├── easy_install-3.7
│   ├── pip
│   ├── pip3
│   ├── pip3.7
│   ├── python -> python3.7
│   ├── python-config
│   ├── python3 -> python3.7
│   ├── python3.7
│   └── wheel
├── include
│   └── python3.7m -> /usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m
└── lib
    └── python3.7
        ├── LICENSE.txt -> /usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/LICENSE.txt
        ├── __future__.py -> /usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/__future__.py
        [...]             Note version-specific paths ^^^^^^^

This is what it looks like to use a broken virtualenv:

% pipenv shell
Warning: Your Pipfile requires python_version 3.7, but you are using None (/Users/m/.local/share/v/l/bin/python).
  $ pipenv --rm and rebuilding the virtual environment may resolve the issue.
  $ pipenv check will surely fail.
Launching subshell in virtual environment…
 . /Users/marti.raudsepp/.local/share/virtualenvs/lemur-N4Rvgxif/bin/activate
%  . /Users/marti.raudsepp/.local/share/virtualenvs/lemur-N4Rvgxif/bin/activate
(lemur) % python
dyld: Library not loaded: @executable_path/../.Python
  Referenced from: /Users/marti.raudsepp/.local/share/virtualenvs/lemur-N4Rvgxif/bin/python
  Reason: image not found
zsh: abort      python

(lemur) % cat $VIRTUAL_ENV/lib/python3.7/__future__.py
cat: /Users/marti.raudsepp/.local/share/virtualenvs/lemur-N4Rvgxif/lib/python3.7/__future__.py: No such file or directory
(lemur) % ls -la $VIRTUAL_ENV/lib/python3.7/__future__.py
lrwxr-xr-x 99 marti.raudsepp 14 Aug 15:14 /Users/marti.raudsepp/.local/share/virtualenvs/lemur-N4Rvgxif/lib/python3.7/__future__.py -> /usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/__future__.py

---

$ pipenv --support

Pipenv version: '2018.11.26'

Pipenv location: '/usr/local/lib/python3.7/site-packages/pipenv'

Python location: '/usr/local/opt/python/bin/python3.7'

Python installations found:

• 3.7.4: /Users/marti.raudsepp/.local/share/virtualenvs/envtest-pJJWjZNx/bin/python3
• 3.7.4: /usr/local/bin/python3
• 3.7.4: /usr/local/bin/python3.7m
• 3.6.1: /usr/local/bin/pypy3
• 2.7.16: /usr/local/bin/python
• 2.7.16: /usr/local/bin/pythonw
• 2.7.10: /usr/bin/python
• 2.7.10: /usr/bin/pythonw
• 2.7.10: /usr/bin/python2.7

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.7.4',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '18.7.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 18.7.0: Tue Aug 20 16:57:14 PDT '
                     '2019; root:xnu-4903.271.2~2/RELEASE_X86_64',
 'python_full_version': '3.7.4',
 'python_version': '3.7',
 'sys_platform': 'darwin'}

[intgr]

fyi, I think this comment expresses the same issue as you effectively?

A little similar perhaps, but not the same. dhouck there is talking about major Python upgrades, such as 3.6 to 3.7.

I’m talking about minor Python patchlevel updates, such as 3.7.3 to 3.7.4 or even Homebrew package re-builds, like 3.7.4 to 3.7.4_1 (which is what I used as example in the original report).

[RNCTX]

Even though it’s a nice interface, I gave up on pipenv because venv is the officially supported virtual environment method upstream. While I was at it, I stopped using the Homebrew python installations in favor of the official python distributions for OSX.

I recreate the convenient pipenv shell commands via bash aliases like…

alias djangodev='set -a; source ~/Google\ Drive/djangodev/.env; set +a; \ 
source ~/.djangodev/bin/activate && cd ~/Google\ Drive/djangodev'