Create a supported "high level" programmatic API for pip

This has come up a few times recently now that pip is available in a standard Python install.

Maybe we should formally support a programmatic API for pip that allows the high-level command line operations pip supports to be run from within a Python interpreter? That may simply mean blessing (and documenting) pip.main() as a supported API.

There may be some odd corner cases to take care with, if we’re considering people running pip.main() from within a persistent interpreter (e.g. IPython or Idle). I’m thinking of cases where sys.modules caches something you’re upgrading, for example. But that may just need documentation saying that those things need an interpreter restart, at least as a starting point.

[pfmoore]

I think there’s enough people talking about using pip from within Python (either prompts or scripts) that we should provide something. Even if it’s very limited, at least that will get people thinking about what exactly they need, and we’ll be able to firm up on details as we go along.

@dstufft for lower-level API’s based around the standards PEPs, we should absolutely be pointing people at packaging.

@RonnyPfannschmidt you have a good point. It’s not at all unreasonable to tell people that to call pip they should be using

subprocess.check_call([sys.executable, '-m', 'pip', ...])

If that’s not sufficient for them (for example, the format of stdout isn’t useful for parsing) we can discuss options (for example, adding a --json argument to informational commands).

On the other hand, I think the big issue is that people expect to be able to do

pip.main(['install', 'foo'])
import foo

from within (say) an Idle session. This is actually no different in principle from running pip in a separate command window and not restarting your Idle session (and indeed it’s exactly the same if you use subprocess) but maybe just documenting that is sufficient - after all, it’s not really a pip issue that you have to take care if you change the contents of sys.path while your Python process is running.

I think I’ll write up a new section for the pip documentation on the basis of subprocess being pip’s supported API. That can be a starting point for people wanting to integrate pip into IDEs, and we can then address any issues as people find them.

[pradyunsg]

That is appropriate, as long as folks using it don’t expect it to “just work”. Further, as noted in the README of that package:

The authors of pip do not condone the use of this package. Relying on pip’s internals is a dangerous idea for your software as they are broken intentionally and regularly. This package may not always be completely updated [snip], so relying on it may break your code! User beware!