Options to improve Python runtime autodiscovery on Windows

Pipenv currently has two Python runtime discovery and/or installation strategies:

1. Search PATH (in pipenv.cli.find_a_system_python), adding any pyenv environments to the end before it does so
2. Install it with pyenv (in pipenv.cli.ensure_python)

This misses a few potential possibilities for improvement on Windows:

1. Discovery based on the Windows launcher configuration file (https://docs.python.org/3/using/windows.html#customization)
2. Discovery based on the Windows registry (https://github.com/zooba/pep514tools#usage)
3. Supporting conda based automatic Python installation in addition to pyenv installation
4. Supporting @uranusjr’s https://github.com/uranusjr/pythonup-windows (and its macOS counterpart)

Actually doing this would probably requiring refactoring the autodiscovery code though, such that the discovery & installation strategy used by ensure_python can be chosen at import time based on the platform.

[uranusjr]

A summary of the current situation:

• PATH-based discovery and pyenv discovery is unchanged.
• A py.exe-based discovery has been implemented on Windows. As long as py.exe is present in PATH, --python=X.Y will consult it to discover Python installations. This has some minor restrictions (not able to discover WOW6432 installations, for example).
• A standalone library, pythonfinder, has been developed aiming to replace the current strategy. The integration is in progress, but is far from being finished. (I feel maybe we should scale the integration effort back a little, and use it only for discovery for now).
• pyenv is still the only option available for auto-installation. I did a minor rewrite in #2572, but there’s no change in functionality.

[uranusjr]

@ncoghlan Wow, I’ve been thinking about the same refactoring as well, and even came up with the same submodule name 😅 Glad to know I was on the right track.

The problem I have with the current implementation it that, it might not be able to integrate PEP 514. If I read the PEP correctly, it allows a distribution to use an arbitrary executable name (with InstallPath\ExecutablePath). For PythonCore (i.e. installers from python.org) it is always InstallPath + \python.exe, but that might not be the case for other distributions. I’m not aware of any distributions/implementations using a different executable name (Anaconda also uses python.exe, and pypy doesn’t implement PEP 514 at all), but it might be better to be more future-proof.

My thought is to implement multiple finders:

• pipenv.pythonfinders.GlobalPythonFinder: Look for python, pythonX, and pythonX.Y in PATH, and call it to check whether it has the right version.
• pipenv.pythonfinders.PyEnvFinder: Look for a matching version from pyenv.
• pipenv.pythonfinders.PEP514Finder: Look for a matching version in the registry.

When the user calls get_runtime_path(), it detects what finder(s) to use. If pyenv is installed (i.e. the right environs are set), use PyEnvFinder and GlobalPythonFinder; if on Windows, use PEP514Finder and GlobalPythonFinder, etc. This way we don’t need to rely on the PATH environ to find the right Python, and I think is the right level of abstraction.

P.S. In regard of install_runtime, I’m currently working on a Python version-management tool for Windows (uranusjr/snafu). The idea is to abstract away the download, install, configure part of installing Python on Windows, so the user can manage multiple Python installations with minimal fuss. It plays a similar role as pyenv, but instead of building just use the official binaries. This could be useful if Pipenv wants to provide auto installation on Windows.