Notes on pipsi compatibility & usability challenges

Kenneth recently dropped the pipsi based installation instructions, presumably because they were causing confusion (https://github.com/pypa/pipenv/commit/65a2106984ec3c9361cc8f8817388b10d69dd861#diff-62c330cd8da31d476bb63b51bb8b1307).

As far as I’m aware, there are three main ways for things to go wrong with pipsi:

1. The same PATH bootstrapping issues that affect all Python CLI applications

See https://github.com/pypa/python-packaging-user-guide/issues/396 for more discussion of that problem.

1. Getting it to use the right Python version

If you use python3 -m pip --user pipsi (or an equivalent) to bootstrap it, then it will default to creating Python 3.x virtual environments. However, if pipsi ends up being installed with Python 2.7, then it will also default to that for all of the applications it installs (which is a hassle for pipenv, since it really prefers to have Python 3.x as the host Python).

1. Getting the pew CLI installed correctly

Even if folks don’t run into the above two issues, pipsi install pipenv doesn’t quite work as desired, because pipenv’s shell integration also has a CLI dependency on pew, not just a Python API dependency.

That distinction doesn’t matter for a regular pipenv installation with pip, since that installs everything to a common destination, so the pew CLI automatically comes along with the Python level dependency declaration.

However, it doesn’t work for pipsi, since breaking the per-app venv isolation is an extra step in the installation process, so the installation currently needs to be spelled as pipsi install pew && pipsi install pipenv.

This isn’t an urgent issue, but I think it could be a good design driver for adding “CLI dependency declaration” support to the packaging metadata formats, as we some clear constraints that we’d like to adhere to in solving this problem:

• updating pipenv to publish new CLI dependency declaration metadata would be fine
• updating pipsi to consume new CLI dependency declaration metadata would be fine
• requiring updates to pew to make things work would not be fine (since it creates a major barrier to adoption if a dependency management feature doesn’t already work with previously published packages - a concrete example of that would be PEP 345’s failed attempt to move away from the de facto standard established by setuptools for spelling dependency declarations vs PEP 508’s pragmatic adoption of the established setuptools conventions)

https://github.com/pypa/interoperability-peps/pull/30/files would be an interesting proposal to consider on this front, as it aimed to tackle the problem of declaring external dependencies in general. As far as I can tell, given suitable adjustments to pipsi, it could potentially work for pipenv's use case as long as external requirements were listed in a separate field:

• pipenv would add bin!pew as a new external requirement
• pip and other existing installers would ignore it, since it would be in a different field added by metadata 2.2
• pipsi could be updated to check for the external binary dependencies listed in the installed project, and if they weren’t already available on the system, check if they could be exported from one of the app’s declared Python level dependencies.

[uranusjr]

I just did a quick grep on the source and it seems we are only using Pew for four things at this point:

1. Create virtualenvs (new_cmd)
2. Global site packages toggle (toggleglobalsidepackages)
3. List virtual environments, to find the one matching the current project (lsenvs)
4. Shell (workon_cmd)

The first three are all quite trivial, and the fourth is already very heavily patched by Pipenv, and has some more patching planned. I feel maybe it is time to just drop dependency to Pew altogether. Without its presence Pipsi compatibility would largely fix itself. We can suggest users to install Pipenv with pipsi install pipenv --python=python3 to make sure it always runs on Python 3.

[uranusjr]

I believe there are also references to pewtwo, which is a vendored version of pew, but Pipsi should be able to pick it up since it is published in Pipenv’s setup.py.