pip should support custom authentication handlers for private pypi

• Pip version: 9.0.1
• Python version: 2.7.13
• Operating system: MacOS X Sierra 10.12.4 *

* any OS, really

Description:

This is a feature request.

It would be super-awesome++ if pip supported custom authentication handler configuration so private pypi repositories are not restricted to http basic auth only. Basically, make MultiDomainBasicAuth the default and no longer the ONLY option in a PipSession as it is today: https://github.com/pypa/pip/blob/9.0.1/pip/download.py#L331-L332

This limitation prevents easy integration with stronger authentication (e.g. 2-way TLS, 2FA, etc.) and SSO schemes at enterprises with private pypi repositories. The lack of support makes basic auth credential distribution and leaking unnecessarily difficult problems to address and combat.

[pfmoore]

Things I’d want to see in any concrete proposal to handle this:

1. A means whereby it’s user-expandable, so that tools like pip don’t need to add new code every time someone comes up with a new protocol/handler/whatever.
2. A way of addressing the bootstrapping issue (user can’t install the handler because they need pip to do so, and pip can’t install without the handler).
3. A reusable solution that will work across PyPA tools, so we can avoid having to implement the same feature (possibly with annoying subtle differences) in pip and twine and …
4. A clarification of how this fits with the fact that pip has no supported programming API, so any sort of plugin cannot rely on anything about pip’s internals remaining constant. (As a practical example, what if we decided to switch from requests to httpx for our network protocol? It’s not impossible that we would do this…)
5. Good documentation and tests for all of the above.

Reasons I think these are important:

1. These same points come up every time we discuss issues like this. For example, the bootstrapping issue came up with the keyring implementation, and wasn’t completely addressed there, so that feature is less useful to some people than it might otherwise be. Let’s not repeat that.
2. Design issues like this are much harder than “just writing the code”, and result in maintenance issues longer term if we just accept a PR without considering them.
3. The interactions between new features for pip and existing features have the potential to become very complex very quickly, and generally when a PR is developed with a focus on just addressing the initial use-case, these interactions are not noticed until after the PR has landed (and often, not until people have started relying on details of the interactions which weren’t ever intended). Again, that can be a maintenance issue, making refactoring of pip’s code base way harder than we can deal with.
4. Test infrastructure for this sort of environment generally doesn’t exist in open source CI offerings, so it’s really hard to ensure adequate testing.

It’s really hard to thrash out this sort of “wider issue” in the context of an open source issue tracker/pull request workflow. That’s where a funded project, with a clear scope and a remit to look at the broad implications, is a potential way forward for proposals like this. And where the use case is specifically around “corporate” infrastructure like private repositories, some sort of funding can help bridge the gap between volunteer resources who have no “itch to scratch” in this area, and businesses that depend on such support but don’t otherwise have a means to influence what features get accepted.

Remember, the pip developer team consists of a very small number of wholly volunteer contributors. We’re working on trying to make things more sustainable, but in the meantime we have to be careful how we manage feature additions. Funded developments is one way we’re exploring of doing this.

(And yes, I understand that the above makes something that “seems simple” into quite a big project. I don’t apologise for that - changes to pip can have a huge impact, and we owe it to all of our users to do our best to ensure they are well managed).

[uranusjr]

I’ve cleaned up the previous comments a bit to focus this thread on the remaining this at hand: implementing a way to plug in custom authentication backends, to support using methods such as Kerberos (#6708) and Windows Integrated Authentication (#8163).

The solution will likely be some kind of a plug-in system, so a user can install a backend alongside with pip, and use a flag to tell pip to use that. So the next questions from what I can tell would be to a) come up with a design, and b) identify places that need to be pluggable. I’m marking this as deferred till PR since some actual code would likely be the easiest way to kick off the discussion.