virtualenv downloading removed version of pathlib2 instead of latest version - resulting in build errors

Issue

Two days ago the owners of package pathlib2 released a new version (pathlib2-2.3.7.post0) which broke when used with legacy code and python 2.7. After several users raised the issue they rolled back to the previous working version (2.3.6). When I force reinstall virtualenv the other, now removed, version is still being downloaded. (no cache is being used either).

If I’m doing something wrong please let me know, but I’m not sure why an AWS instance in an autoscale group with a forced reinstall would download and use a version that is not the latest on pypi. Thank you.

Stack trace of BuildKite build: (you can see pathlib2-2.3.7.post0 being used)

Running commands | 6s
-- | --
  | [2022-02-10T06:32:22Z] $ [[ -d '/tmp/env' ]] && exit 0 \|\| pip install --force-reinstall --no-cache-dir --user virtualenv && ~/.local/bin/virtualenv /tmp/env && source /tmp/env/bin/activate && pip install --force-reinstall --no-cache-dir -r requirements/deploy.txt
  | [2022-02-10T06:32:24Z] Collecting virtualenv
  | [2022-02-10T06:32:24Z]   Downloading https://files.pythonhosted.org/packages/99/0c/26206e19ab4176532dc0509e67e4d0cc8e6a82eab204e7e4ceff2664769a/virtualenv-20.13.1-py2.py3-none-any.whl (8.6MB)
  | 100% \|████████████████████████████████\| 8.6MB 73.2MB/s
  | [2022-02-10T06:32:25Z] Collecting importlib-metadata>=0.12; python_version < "3.8" (from virtualenv)
  | [2022-02-10T06:32:25Z]   Downloading https://files.pythonhosted.org/packages/cf/b4/877779cd7b5a15536ecbe0655cfb35a0de0ede6d888151fd7356d278c47d/importlib_metadata-2.1.3-py2.py3-none-any.whl
  | [2022-02-10T06:32:25Z] Requirement already satisfied: six<2,>=1.9.0 in /usr/lib/python2.7/site-packages (from virtualenv)
  | [2022-02-10T06:32:25Z] Collecting filelock<4,>=3.2 (from virtualenv)
  | [2022-02-10T06:32:25Z]   Downloading https://files.pythonhosted.org/packages/f1/12/585c4d8bef61cd7f9497a021d935ba9fb3127f5e45593c2fcc313897f582/filelock-3.2.1-py2.py3-none-any.whl
  | [2022-02-10T06:32:25Z] Collecting importlib-resources>=1.0; python_version < "3.7" (from virtualenv)
  | [2022-02-10T06:32:25Z]   Downloading https://files.pythonhosted.org/packages/8d/94/2f6ceee0c4e63bff0177c07e68d27c937a19f6bc77c4739755b49f5adb04/importlib_resources-3.3.1-py2.py3-none-any.whl
  | [2022-02-10T06:32:25Z] Collecting platformdirs<3,>=2 (from virtualenv)
  | [2022-02-10T06:32:25Z]   Downloading https://files.pythonhosted.org/packages/12/57/f4e6d433fba8bde239baa65f626a31fc6d0556a2065fcc81aaa0f4f1507b/platformdirs-2.0.2-py2.py3-none-any.whl
  | [2022-02-10T06:32:25Z] Collecting distlib<1,>=0.3.1 (from virtualenv)
  | [2022-02-10T06:32:25Z]   Downloading https://files.pythonhosted.org/packages/ac/a3/8ee4f54d5f12e16eeeda6b7df3dfdbda24e6cc572c86ff959a4ce110391b/distlib-0.3.4-py2.py3-none-any.whl (461kB)
  | 100% \|████████████████████████████████\| 471kB 72.5MB/s
  | [2022-02-10T06:32:26Z] Collecting pathlib2<3,>=2.3.3; python_version < "3.4" and sys_platform != "win32" (from virtualenv)
  | [2022-02-10T06:32:26Z]   Downloading https://files.pythonhosted.org/packages/59/b7/6cb6e5cb66d934b9f3d51761ceb4a52bcdd4446048b9107e7ebb60b8a04c/pathlib2-2.3.7.post0-py2.py3-none-any.whl
  | [2022-02-10T06:32:26Z] Collecting contextlib2; python_version < "3" (from importlib-metadata>=0.12; python_version < "3.8"->virtualenv)
  | [2022-02-10T06:32:26Z]   Downloading https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
  | [2022-02-10T06:32:26Z] Collecting zipp>=0.5 (from importlib-metadata>=0.12; python_version < "3.8"->virtualenv)
  | [2022-02-10T06:32:26Z]   Downloading https://files.pythonhosted.org/packages/96/0a/67556e9b7782df7118c1f49bdc494da5e5e429c93aa77965f33e81287c8c/zipp-1.2.0-py2.py3-none-any.whl
  | [2022-02-10T06:32:26Z] Collecting configparser>=3.5; python_version < "3" (from importlib-metadata>=0.12; python_version < "3.8"->virtualenv)
  | [2022-02-10T06:32:26Z]   Downloading https://files.pythonhosted.org/packages/7a/2a/95ed0501cf5d8709490b1d3a3f9b5cf340da6c433f896bbe9ce08dbe6785/configparser-4.0.2-py2.py3-none-any.whl
  | [2022-02-10T06:32:26Z] Collecting typing; python_version < "3.5" (from importlib-resources>=1.0; python_version < "3.7"->virtualenv)
  | [2022-02-10T06:32:26Z]   Downloading https://files.pythonhosted.org/packages/0b/cb/da856e81731833b94da70a08712f658416266a5fb2a9d9e426c8061becef/typing-3.10.0.0-py2-none-any.whl
  | [2022-02-10T06:32:26Z] Collecting singledispatch; python_version < "3.4" (from importlib-resources>=1.0; python_version < "3.7"->virtualenv)
  | [2022-02-10T06:32:26Z]   Downloading https://files.pythonhosted.org/packages/3f/09/2a17c6505a8373229cb3de6161e3af74ff90afcf4516719aa6cf32fad151/singledispatch-3.7.0-py2.py3-none-any.whl
  | [2022-02-10T06:32:27Z] Collecting scandir; python_version < "3.5" (from pathlib2<3,>=2.3.3; python_version < "3.4" and sys_platform != "win32"->virtualenv)
  | [2022-02-10T06:32:27Z]   Downloading https://files.pythonhosted.org/packages/df/f5/9c052db7bd54d0cbf1bc0bb6554362bba1012d03e5888950a4f5c5dadc4e/scandir-1.10.0.tar.gz
  | [2022-02-10T06:32:27Z] Installing collected packages: contextlib2, zipp, typing, scandir, pathlib2, configparser, importlib-metadata, filelock, singledispatch, importlib-resources, platformdirs, distlib, virtualenv
  | [2022-02-10T06:32:27Z]   Running setup.py install for scandir ... done
  | [2022-02-10T06:32:28Z] Successfully installed configparser-4.0.2 contextlib2-0.6.0.post1 distlib-0.3.4 filelock-3.2.1 importlib-metadata-2.1.3 importlib-resources-3.3.1 pathlib2-2.3.7.post0 platformdirs-2.0.2 scandir-1.10.0 singledispatch-3.7.0 typing-3.10.0.0 virtualenv-20.13.1 zipp-1.2.0
  | [2022-02-10T06:32:28Z] Traceback (most recent call last):
  | [2022-02-10T06:32:28Z]   File "/var/lib/buildkite-agent/.local/bin/virtualenv", line 7, in <module>
  | [2022-02-10T06:32:28Z]     from virtualenv.__main__ import run_with_catch
  | [2022-02-10T06:32:28Z]   File "/var/lib/buildkite-agent/.local/lib/python2.7/site-packages/virtualenv/__init__.py", line 3, in <module>
  | [2022-02-10T06:32:28Z]     from .run import cli_run, session_via_cli
  | [2022-02-10T06:32:28Z]   File "/var/lib/buildkite-agent/.local/lib/python2.7/site-packages/virtualenv/run/__init__.py", line 7, in <module>
  | [2022-02-10T06:32:28Z]     from ..app_data import make_app_data
  | [2022-02-10T06:32:28Z]   File "/var/lib/buildkite-agent/.local/lib/python2.7/site-packages/virtualenv/app_data/__init__.py", line 12, in <module>
  | [2022-02-10T06:32:28Z]     from .read_only import ReadOnlyAppData
  | [2022-02-10T06:32:28Z]   File "/var/lib/buildkite-agent/.local/lib/python2.7/site-packages/virtualenv/app_data/read_only.py", line 3, in <module>
  | [2022-02-10T06:32:28Z]     from virtualenv.util.lock import NoOpFileLock
  | [2022-02-10T06:32:28Z]   File "/var/lib/buildkite-agent/.local/lib/python2.7/site-packages/virtualenv/util/lock.py", line 13, in <module>
  | [2022-02-10T06:32:28Z]     from virtualenv.util.path import Path
  | [2022-02-10T06:32:28Z]   File "/var/lib/buildkite-agent/.local/lib/python2.7/site-packages/virtualenv/util/path/__init__.py", line 3, in <module>
  | [2022-02-10T06:32:28Z]     from ._pathlib import Path
  | [2022-02-10T06:32:28Z]   File "/var/lib/buildkite-agent/.local/lib/python2.7/site-packages/virtualenv/util/path/_pathlib/__init__.py", line 14, in <module>
  | [2022-02-10T06:32:28Z]     from pathlib2 import Path
  | [2022-02-10T06:32:28Z]   File "/var/lib/buildkite-agent/.local/lib/python2.7/site-packages/pathlib2/__init__.py", line 26, in <module>
  | [2022-02-10T06:32:28Z]     from six.moves.collections_abc import Sequence
  | [2022-02-10T06:32:28Z] ImportError: No module named collections_abc
  | [2022-02-10T06:32:28Z] 🚨 Error: The command exited with status 1

[gaborbernat]

But they fixed their issue - this occurs when I install virtualenv - which uses a cached version of pathlib2?

The installation is not handled by the virtualenv project but by the pip project. So you’d need to enquire there.

[gaborbernat]

Duplicate of https://github.com/pypa/virtualenv/issues/2301