Implement PEP 376 REQUESTED

I’m opening this to discuss the opportunity to implement https://www.python.org/dev/peps/pep-0376/#requested. I’m willing to work on it, if there is agreement about its usefulness.

What’s the problem this feature will solve?

It is currently not possible to discover, in the database of installed distribution, which distributions were installed by the user and which distributions were installed as dependencies.

Implementing the REQUESTED file would enable use cases such as smart uninstall (e.g. informing the users about possibly unneeded distributions) #5823

Describe the solution you’d like

Generate the REQUESTED file in pip install for top level requirements, i.e. provided by the user on the command line and packages provided via requirements files (-r option).

In a second stage, the freeze command could evolve a mechanism to output top level requirements on the one hand, and constraints for dependencies on the other hand, enabling a pip freeze | pip install loop that would preserve REQUESTEDness of distributions.

Alternative Solutions

Some higher level tools such as poetry implement this via their own requirement/lockfile format. pip-tools seems to achieve a similar goal with comments in the requirements file.

Having this standardized metadata piece implemented by pip would simplify the creation of such higher level tools, in addition to enabling additional pip use cases.

Note: flit, when used as an installer, implements the REQUESTED file.

[sbidoul]

Pip works at a much lower level - basically package-by-package (with packages triggering others as dependencies, but pip still treats each one mostly independently).

REQUESTED is simply a way to mark individual packages as being manually installed. Similar to what apt-mark manual does in debian. As such, it remains a package-by-package information.

how do we deal with --target and --prefix

Wrt this very specific goal of marking manually installed package, --target and --prefix do not make any difference, unless I’m missing something.

[chrahunt]

I mentioned tracking what is installed directly in #7807 (comment). I think it’s a good idea.

User later wants to use child in its own right - how to say "mark child as requested, but don’t change anything else?

pip install child?

This seems reasonable to me, and is what I would expect. apt does something similar too - when you install something that’s already installed it gets marked as manually installed.

3\. Later still, user wants to drop the "requested" marker on `child`, and have it go back to being considered as "just" a dependency of `parent`.

We don’t necessarily have to decide this in the initial implementation - users can delete the REQUESTED file manually.