pipx enters in infinite loop when trying to install a package on python distributed with MSYS2 on Windows

Describe the bug When using MSYS2, package mingw-w64-x86_64-python (https://packages.msys2.org/package/mingw-w64-x86_64-python), pipx enters in infinite loop when installing a package.

pipx install pycowsay --verbose
pipx >(setup:717): pipx version is 0.16.3
pipx >(setup:718): Default python interpreter is 'c:/users/user1/software/tools/msys64/mingw64/bin/python3.exe'
pipx >(package_name_from_spec:323): Determined package name: pycowsay
pipx >(package_name_from_spec:324): Package name determined in 0.0s
creating virtual environment...
pipx >(run_subprocess:135): running c:/users/user1/software/tools/msys64/mingw64/bin/python3.exe -m venv --without-pip C:/Users/user1/.local/pipx/venvs/pycowsay
creating shared libraries...
pipx >(run_subprocess:135): running c:/users/user1/software/tools/msys64/mingw64/bin/python3.exe -m venv --clear C:/Users/user1/.local/pipx/shared
creating shared libraries...
pipx >(run_subprocess:135): running c:/users/user1/software/tools/msys64/mingw64/bin/python3.exe -m venv --clear C:/Users/user1/.local/pipx/shared
creating shared libraries...
pipx >(run_subprocess:135): running c:/users/user1/software/tools/msys64/mingw64/bin/python3.exe -m venv --clear C:/Users/user1/.local/pipx/shared
creating shared libraries...
pipx >(run_subprocess:135): running c:/users/user1/software/tools/msys64/mingw64/bin/python3.exe -m venv --clear C:/Users/user1/.local/pipx/shared
creating shared libraries...
pipx >(run_subprocess:135): running c:/users/user1/software/tools/msys64/mingw64/bin/python3.exe -m venv --clear C:/Users/user1/.local/pipx/shared
creating shared libraries...
pipx >(run_subprocess:135): running c:/users/user1/software/tools/msys64/mingw64/bin/python3.exe -m venv --clear C:/Users/user1/.local/pipx/shared
(...)

if I execute manually the command python3.exe -m venv --clear C:/Users/user1/.local/pipx/shared it is executed successfully

c:/users/user1/software/tools/msys64/mingw64/bin/python3.exe -m venv --clear C:/Users/user1/.local/pipx/shared
Looking in links: c:\Users\user1\AppData\Local\Temp\tmphnr44wk9
Processing c:\users\user1\appdata\local\temp\tmphnr44wk9\setuptools-49.2.1-py3-none-any.whl
Processing c:\users\user1\appdata\local\temp\tmphnr44wk9\pip-20.2.3-py2.py3-none-any.whl
Installing collected packages: setuptools, pip
Successfully installed pip-20.2.3 setuptools-49.2.1

Contents of pipx logs:

   381.0ms (setup:715): 2021-06-02 20:15:15
   381.0ms (setup:716): C:\Users\user1\.local\bin\pipx install pycowsay
   381.0ms (setup:717): pipx version is 0.16.3
   381.0ms (setup:718): Default python interpreter is 'c:/users/user1/software/tools/msys64/mingw64/bin/python3.exe'
   383.0ms (package_name_from_spec:323): Determined package name: pycowsay
   383.0ms (package_name_from_spec:324): Package name determined in 0.0s
   385.0ms (run_subprocess:135): running c:/users/user1/software/tools/msys64/mingw64/bin/python3.exe -m venv --without-pip C:/Users/user1/.local/pipx/venvs/pycowsay
   843.1ms (run_subprocess:148): stdout:
   844.1ms (run_subprocess:150): stderr:
   844.1ms (run_subprocess:151): returncode: 0
   846.1ms (run_subprocess:135): running c:/users/user1/software/tools/msys64/mingw64/bin/python3.exe -m venv --clear C:/Users/user1/.local/pipx/shared
 16771.8ms (run_subprocess:148): stdout: Looking in links: c:\Users\user1\AppData\Local\Temp\tmpxbeu6b31
Processing c:\users\user1\appdata\local\temp\tmpxbeu6b31\setuptools-49.2.1-py3-none-any.whl
Processing c:\users\user1\appdata\local\temp\tmpxbeu6b31\pip-20.2.3-py2.py3-none-any.whl
Installing collected packages: setuptools, pip
Successfully installed pip-20.2.3 setuptools-49.2.1
 16771.8ms (run_subprocess:150): stderr:
 16772.8ms (run_subprocess:151): returncode: 0
 16774.8ms (run_subprocess:135): running c:/users/user1/software/tools/msys64/mingw64/bin/python3.exe -m venv --clear C:/Users/user1/.local/pipx/shared
 34595.2ms (run_subprocess:148): stdout: Looking in links: c:\Users\user1\AppData\Local\Temp\tmpreoye3oe
Processing c:\users\user1\appdata\local\temp\tmpreoye3oe\setuptools-49.2.1-py3-none-any.whl
Processing c:\users\user1\appdata\local\temp\tmpreoye3oe\pip-20.2.3-py2.py3-none-any.whl
Installing collected packages: setuptools, pip
Successfully installed pip-20.2.3 setuptools-49.2.1
 34595.2ms (run_subprocess:150): stderr:
 34595.2ms (run_subprocess:151): returncode: 0
 34597.2ms (run_subprocess:135): running c:/users/user1/software/tools/msys64/mingw64/bin/python3.exe -m venv --clear C:/Users/user1/.local/pipx/shared
 51837.0ms (run_subprocess:148): stdout: Looking in links: c:\Users\user1\AppData\Local\Temp\tmp_75yare0
Processing c:\users\user1\appdata\local\temp\tmp_75yare0\setuptools-49.2.1-py3-none-any.whl
Processing c:\users\user1\appdata\local\temp\tmp_75yare0\pip-20.2.3-py2.py3-none-any.whl
Installing collected packages: setuptools, pip
Successfully installed pip-20.2.3 setuptools-49.2.1
 51838.0ms (run_subprocess:150): stderr:
 51838.0ms (run_subprocess:151): returncode: 0
 (
 (...)

I have seen a similar issue happen with this particular python distribution (MSYS2) and other tools. For example https://github.com/python-poetry/poetry/issues/2867

This distribution has one particularity. Although it is built for Windows, venvs created with it have POSIX-style:

user1@HOST ~/A/L/p/C/v/somevenv> ls -al
total 5
drwxr-xr-x 1 user1 user1   0 Aug 29 23:05 ./
drwxr-xr-x 1 user1 user1   0 Aug 29 23:05 ../
drwxr-xr-x 1 user1 user1   0 Aug 29 23:06 bin/
drwxr-xr-x 1 user1 user1   0 Aug 29 23:05 include/
drwxr-xr-x 1 user1 user1   0 Aug 29 23:05 lib/
-rw-r--r-- 1 user1 user1 106 Aug 29 23:05 pyvenv.cfg

The venvs created with have lib and bin directories instead of Scripts directories. I suspect that pipx may be expecting a Windows-style venv because the underlying OS is Windows. See https://github.com/pypa/pipx/blob/5e26b3b857a568be7fde40a9ef2caa8f7c0c040d/src/pipx/util.py#L79

it is possible to detect this python distro with sysconfig.get_platform() . It will return mingw . See https://stackoverflow.com/questions/35956180/how-to-determine-the-python-is-mingw-or-official-build

How to reproduce

1 - Install MSYS2 on Windows 10 2 - From MSYS shell, install python mingw64 : pacman -S mingw-w64-x86_64-python and pacman -S mingw-w64-x86_64-python-pip 3 - Install pipx using PIP 4 - pipx install pycowsay 5 - The process will never complete and venv will be cleaned and rebuilt in an infinite loop

Expected behavior

Install should complete successfully

[itsayellow]

Thanks for the info.

Sigh so this is some weird mix of Windows and POSIX, because we have the bin/ directory instead of Scripts/, but at the same time what’s inside the bin/ directory are the classic Windows files (e.g. Activate.ps1, and all executables end in *.exe). How confusing.

I have never understood why python on Windows used Scripts/ Instead of bin/ anyway, and now this is weird in a new way. 😛

[cs01]

I don’t think it’s an age detection thing because the output should say something along the lines of how old the shared libs are and whether they are over the age threshold. I think the lib/bin vs Scripts might be the issue, and pipx just never sees what it thinks it should, thus keeps trying to install it.