virtualenv can't be installed as `--user` when an older global installation exists.

Issue

I have the following:

$ virtualenv --version
virtualenv 20.0.18 from /cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/python/3.8.10/lib/python3.8/site-packages/virtualenv/__init__.py
$ virtualenv
usage: virtualenv [--version] [--with-traceback] [-v | -q] [--app-data APP_DATA] [--clear-app-data] [--discovery {builtin}] [-p py] [--creator {builtin,cpython3-posix,venv}] [--seeder {app-data,pip}] [--no-seed]
                  [--activators comma_sep_list] [--clear] [--system-site-packages] [--symlinks | --copies] [--download | --no-download] [--extra-search-dir d [d ...]] [--pip version] [--setuptools version] [--wheel version] [--no-pip]
                  [--no-setuptools] [--no-wheel] [--symlink-app-data] [--prompt prompt] [-h]
                  dest
virtualenv: error: the following arguments are required: dest

if I now try to upgrade virtualenv in my user account:

$ pip install --user --upgrade virtualenv
Ignoring pip: markers 'python_version < "3"' don't match your environment
Looking in links: /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo/avx2, /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo/generic, /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/virtualenv-20.13.1+computecanada-py2.py3-none-any.whl
Collecting filelock<4,>=3.2
  Using cached filelock-3.5.0-py3-none-any.whl (10.0 kB)
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/platformdirs-2.5.0+computecanada-py3-none-any.whl
Requirement already satisfied, skipping upgrade: six<2,>=1.9.0 in /cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/python/3.8.10/lib/python3.8/site-packages (from virtualenv) (1.14.0)
Collecting distlib<1,>=0.3.1
  Using cached distlib-0.3.4-py2.py3-none-any.whl (461 kB)
Installing collected packages: filelock, platformdirs, distlib, virtualenv
Successfully installed distlib-0.3.4 filelock-3.5.0 platformdirs-2.5.0+computecanada virtualenv-20.13.1+computecanada

and then rerun the same:

$ virtualenv --version
virtualenv 20.13.1 from /home/mboisson/.local/lib/python3.8/site-packages/virtualenv/__init__.py
$ virtualenv
ModuleNotFoundError: No module named 'virtualenv.seed.via_app_data'

Environment

Provide at least:

• OS: irrelevant

$ pip list
Package        Version
-------------- ---------------------
appdirs        1.4.3
distlib        0.3.4
filelock       3.5.0
more-itertools 8.2.0
pip            20.0.2
platformdirs   2.5.0+computecanada
setuptools     45.2.0
setuptools-scm 3.5.0
six            1.14.0
virtualenv     20.13.1+computecanada
wheel          0.34.2

I believe this issue is caused by the switch from importlib_metadata to importlib.metadata in PR https://github.com/pypa/virtualenv/pull/2246

In importlib_metadata’s version of entry_points, there is a step to make the imported entry points unique: https://github.com/python/importlib_metadata/blob/main/importlib_metadata/__init__.py#L995

This ensures that only the highest priority entry_points.txt from virtualenv*.dist_info is considered.

This is also the case in importlib.metadata’s latest version: https://github.com/python/cpython/blob/main/Lib/importlib/metadata/__init__.py

but it was not the case in: Python 3.8: https://github.com/python/cpython/blob/3.8/Lib/importlib/metadata.py#L541 Python 3.9: https://github.com/python/cpython/blob/3.9/Lib/importlib/metadata.py#L569 It was not until Python 3.10 that this was taken up: https://github.com/python/cpython/blob/3.10/Lib/importlib/metadata/__init__.py#L978

I believe importlib_metadata should be used for Python < 3.10.

[mboisson]

Not supporting it is simply something not possible in cluster and in shared environments. Having some central packages and some user-packages is common place. This behavior is simply broken for virtualenv. It works for everything else, and it works with python 3.10, so I’m not sure what is the issue.

[gaborbernat]

Not supporting it is simply something not possible in cluster and in shared environments.

pipx installs content into user space virtual environments, so solves the problem better than the --user flag. I know it’s commonplace but it’s not a good solution to the problem. I rather not add support for something that happens to work now, but is brittle in general.