Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Virtualenv needs to be re-created for minor Pyhton updates on Homebrew (a.k.a please use venv!)

See original GitHub issue

Issue description

Python versions 3.3+ bundle a venv module for creating virtualenvs (python -m venv). The virtualenvs created by venv are tidier and supported by Python upstream itself.

There is also a major issue with virtualenv on macOS using Homebrew: it creates symlinks specific to the currently installed package version. This means every time Homebrew updates Python (including minor version updates), I need to recreate all virtualenvs. This gets annoying quickly. venv virtualenvs, however, are not affected by this issue.

Expected result

Virtualenv created with venv:

% python3 -m venv ./venv
% . ./venv/bin/activate
(venv) % tree -L 4 $VIRTUAL_ENV
├── bin
│   ├── activate
│   ├── activate.csh
│   ├──
│   ├── easy_install
│   ├── easy_install-3.7
│   ├── pip
│   ├── pip3
│   ├── pip3.7
│   ├── python -> python3
│   └── python3 -> /usr/local/bin/python3
├── include
├── lib
│   └── python3.7
│       └── site-packages
│           ├── __pycache__
│           ├──
│           ├── pip
│           ├── pip-19.0.3.dist-info
│           ├── pkg_resources
│           ├── setuptools
│           └── setuptools-40.8.0.dist-info
└── pyvenv.cfg

11 directories, 12 files

Actual result + Steps to replicate

 % cd /tmp/envtest
 % pipenv shell
Creating a virtualenv for this project…
Pipfile: /private/tmp/envtest/Pipfile
Using /usr/local/opt/python/bin/python3.7 (3.7.4) to create virtualenv…
⠼ Creating virtual environment...Already using interpreter /usr/local/opt/python/bin/python3.7
Using base prefix '/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7'
New python executable in /Users/marti.raudsepp/.local/share/virtualenvs/envtest-pJJWjZNx/bin/python3.7
Also creating executable in /Users/marti.raudsepp/.local/share/virtualenvs/envtest-pJJWjZNx/bin/python
Installing setuptools, pip, wheel...

✔ Successfully created virtual environment!
Virtualenv location: /Users/marti.raudsepp/.local/share/virtualenvs/envtest-pJJWjZNx
Creating a Pipfile for this project…
Launching subshell in virtual environment…
 . /Users/marti.raudsepp/.local/share/virtualenvs/envtest-pJJWjZNx/bin/activate
%  . /Users/marti.raudsepp/.local/share/virtualenvs/envtest-pJJWjZNx/bin/activate
(envtest) % tree $VIRTUAL_ENV -L 3
├── bin
│   ├── activate
│   ├── activate.csh
│   ├──
│   ├── activate.ps1
│   ├── activate.xsh
│   ├──
│   ├── easy_install
│   ├── easy_install-3.7
│   ├── pip
│   ├── pip3
│   ├── pip3.7
│   ├── python -> python3.7
│   ├── python-config
│   ├── python3 -> python3.7
│   ├── python3.7
│   └── wheel
├── include
│   └── python3.7m -> /usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m
└── lib
    └── python3.7
        ├── LICENSE.txt -> /usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/LICENSE.txt
        ├── -> /usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/
        [...]             Note version-specific paths ^^^^^^^

This is what it looks like to use a broken virtualenv:

% pipenv shell
Warning: Your Pipfile requires python_version 3.7, but you are using None (/Users/m/.local/share/v/l/bin/python).
  $ pipenv --rm and rebuilding the virtual environment may resolve the issue.
  $ pipenv check will surely fail.
Launching subshell in virtual environment…
 . /Users/marti.raudsepp/.local/share/virtualenvs/lemur-N4Rvgxif/bin/activate
%  . /Users/marti.raudsepp/.local/share/virtualenvs/lemur-N4Rvgxif/bin/activate
(lemur) % python
dyld: Library not loaded: @executable_path/../.Python
  Referenced from: /Users/marti.raudsepp/.local/share/virtualenvs/lemur-N4Rvgxif/bin/python
  Reason: image not found
zsh: abort      python

(lemur) % cat $VIRTUAL_ENV/lib/python3.7/
cat: /Users/marti.raudsepp/.local/share/virtualenvs/lemur-N4Rvgxif/lib/python3.7/ No such file or directory
(lemur) % ls -la $VIRTUAL_ENV/lib/python3.7/
lrwxr-xr-x 99 marti.raudsepp 14 Aug 15:14 /Users/marti.raudsepp/.local/share/virtualenvs/lemur-N4Rvgxif/lib/python3.7/ -> /usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/

$ pipenv --support

Pipenv version: '2018.11.26'

Pipenv location: '/usr/local/lib/python3.7/site-packages/pipenv'

Python location: '/usr/local/opt/python/bin/python3.7'

Python installations found:

  • 3.7.4: /Users/marti.raudsepp/.local/share/virtualenvs/envtest-pJJWjZNx/bin/python3
  • 3.7.4: /usr/local/bin/python3
  • 3.7.4: /usr/local/bin/python3.7m
  • 3.6.1: /usr/local/bin/pypy3
  • 2.7.16: /usr/local/bin/python
  • 2.7.16: /usr/local/bin/pythonw
  • 2.7.10: /usr/bin/python
  • 2.7.10: /usr/bin/pythonw
  • 2.7.10: /usr/bin/python2.7

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.7.4',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '18.7.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 18.7.0: Tue Aug 20 16:57:14 PDT '
                     '2019; root:xnu-4903.271.2~2/RELEASE_X86_64',
 'python_full_version': '3.7.4',
 'python_version': '3.7',
 'sys_platform': 'darwin'}

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Reactions:1
  • Comments:8 (1 by maintainers)

github_iconTop GitHub Comments

intgrcommented, Nov 13, 2019

fyi, I think this comment expresses the same issue as you effectively?

A little similar perhaps, but not the same. dhouck there is talking about major Python upgrades, such as 3.6 to 3.7.

I’m talking about minor Python patchlevel updates, such as 3.7.3 to 3.7.4 or even Homebrew package re-builds, like 3.7.4 to 3.7.4_1 (which is what I used as example in the original report).

RNCTXcommented, Jan 18, 2020

Even though it’s a nice interface, I gave up on pipenv because venv is the officially supported virtual environment method upstream. While I was at it, I stopped using the Homebrew python installations in favor of the official python distributions for OSX.

I recreate the convenient pipenv shell commands via bash aliases like…

alias djangodev='set -a; source ~/Google\ Drive/djangodev/.env; set +a; \ 
source ~/.djangodev/bin/activate && cd ~/Google\ Drive/djangodev'
Read more comments on GitHub >

github_iconTop Results From Across the Web

venv — Creation of virtual environments — Python 3.11.1 ...
Source code: Lib/venv/ The venv module supports creating lightweight “virtual environments”, each with their own independent set of Python packages ...
Read more >
Upgrade python in a virtualenv - pip - Stack Overflow
Per the docs: Upgrade the environment directory to use this version of Python, assuming Python has been upgraded in-place. python3 -m venv --upgrade...
Read more >
Virtualenv and venv: Python virtual environments explained
Take advantage of virtual environments in Python to create and manage separate environments for your Python projects.
Read more >
Using Virtual Environments - Python I
Creating a python virtual environment · Open a terminal window and create an empty directory somewhere–I called mine “using-python-venv”–and cd ...
Read more >
How to Setup a Python Virtual Environment - Medium
Install Homebrew. If you aren't already using Homebrew on your Mac, you should consider it. Homebrew is a package manager for macOS that...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found