pip >= 20.0 fails when running in Jython on Windows due to dependency on JNA

Environment

• pip version: 20.0 - 20.2.2
• Python version: Jython 2.7.2 (Java 11)
• OS: Windows 10

Description

Updating pip on Jython 2.7.2 from 19.1 to the latest causes pip to stop working.

How to Reproduce

1. Install Jython
2. Run “jython -m ensurepip” to install pip 19.1
3. Run “jython -m pip install --upgrade pip” to go to pip 20.2.2
4. Run “jython -m pip install <anything>” and observe the following error.

Output

jython -m pip install requests
Traceback (most recent call last):
  File "C:\jython2.7.2\Lib\runpy.py", line 161, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\jython2.7.2\Lib\runpy.py", line 72, in _run_code
    exec code in run_globals
  File "C:\jython2.7.2\Lib\site-packages\pip\__main__.py", line 23, in <module>
    from pip._internal.cli.main import main as _main  # isort:skip # noqa
  File "C:\jython2.7.2\Lib\site-packages\pip\_internal\cli\main.py", line 10, in <module>
    from pip._internal.cli.autocompletion import autocomplete
  File "C:\jython2.7.2\Lib\site-packages\pip\_internal\cli\autocompletion.py", line 9, in <module>
    from pip._internal.cli.main_parser import create_main_parser
  File "C:\jython2.7.2\Lib\site-packages\pip\_internal\cli\main_parser.py", line 7, in <module>
    from pip._internal.cli import cmdoptions
  File "C:\jython2.7.2\Lib\site-packages\pip\_internal\cli\cmdoptions.py", line 23, in <module>
    from pip._internal.cli.progress_bars import BAR_TYPES
  File "C:\jython2.7.2\Lib\site-packages\pip\_internal\cli\progress_bars.py", line 12, in <module>
    from pip._internal.utils.logging import get_indentation
  File "C:\jython2.7.2\Lib\site-packages\pip\_internal\utils\logging.py", line 18, in <module>
    from pip._internal.utils.misc import ensure_dir
  File "C:\jython2.7.2\Lib\site-packages\pip\_internal\utils\misc.py", line 33, in <module>
    from pip._internal.locations import (
  File "C:\jython2.7.2\Lib\site-packages\pip\_internal\locations.py", line 31, in <module>
    USER_CACHE_DIR = appdirs.user_cache_dir("pip")
  File "C:\jython2.7.2\Lib\site-packages\pip\_internal\utils\appdirs.py", line 23, in user_cache_dir
    return _appdirs.user_cache_dir(appname, appauthor=False)
  File "C:\jython2.7.2\Lib\site-packages\pip\_vendor\appdirs.py", line 300, in user_cache_dir
    path = os.path.normpath(_get_win_folder("CSIDL_LOCAL_APPDATA"))
  File "C:\jython2.7.2\Lib\site-packages\pip\_vendor\appdirs.py", line 474, in _get_win_folder_from_registry
    import _winreg
ImportError: No module named _winreg

Analysis

The ImportError is a bit of a red herring.

This seems to be due to the relatively recent appdirs refactor, which now points to _vendor for most code, making it use the CSIDL directory names in Windows, and using JNA to look those up when using Java.

The problem is that JNA doesn’t ship with Java, so if someone is running Jython from the command line, they need to download two JNA jars and add them as command line arguments whenever they invoke pip. As mentioned, the actual error message is a red herring, since _winreg is only attempted after pip tries JNA, but at that point JNA has failed. (Jython doesn’t include _winreg, so this will always fail.)

This seems important to resolve this year, since Jython 2.7 is the current version, 3.x is still a while out, and it’s likely folks running Jython 2.7 will be upgrading to pip <20.latest> going into the future, as pip 21 will stop supporting Python 2.7. Getting a resolution into pip 20 before pip 21 comes around would be ideal.

Possible resolutions

1. Restore the old behavior as a fallback for _get_win_folder when everything else fails. ( Use os.path.expanduser(“~/.cache”) and “~/.config”. )
2. Create a slightly more complex function as a fallback. That could use os.path to get the user directory, and then have a mapping of the CSIDL to directory paths, to attempt to return the right path for the current Windows version and roaming profile.
3. Change the OS detection back to be similar to how _internal/tools/appdirs.py did it. WINDOWS was used as the variable to determine if the OS was on Windows, and Jython returned False, making it just go through the code paths that use os.path.expanduser() by default. (Doing this would bypass the JNA code, which is a nice piece of code for folks running systems with JNA, so it might not be ideal.)
4. Although less desirable, throwing an Exception if the user is running Jython and doesn’t have JNA would at least help folks troubleshoot, instead of being presented with a _winreg module import error.
5. Something I haven’t thought of?

If there’s appetite for any of these options, I am happy to submit a PR if that would be helpful. If so, please let me know which of the options makes the most sense.

[uranusjr]

Can you also report this upstream to appdirs? It would be better to push the change upstream if we’re able to develop a more resillient logic. The upstream maintainers are also in a better position to make a decision on this than pip developers.

[pradyunsg]

a few pip specific changes, so I don’t want to step on any toes

pip-specific patches to appdirs can be found here: https://github.com/pypa/pip/blob/master/tools/automation/vendoring/patches/appdirs.patch

If you want to update the vendored version (we only vendor released versions), you’d want to update the vendor.txt file and then run tox -e vendoring.