Reject packages without a Requires-Python

A problem I’ve come across many times now is that a package will drop support for an old version and they’ll even add a python_requires directive to their setup.py, only to cut a release using an old version of setuptools that doesn’t actually understand the python_requires directive and thus silently fails. See, e.g. sphinx. This causes a bunch of problems because the erroneously uploaded packages now pollute pip installs of that package unless they are removed from PyPI or a post release is made.

Since there is no good reason to upload a package without explicitly specifying a Requires-Python directive, I think we can assume that this is an error and reject packages that don’t have a Requires-Python.

My recommendation for migrating to the “always throw an error” version:

1. Start throwing a warning when Requires-Python is missing, with a link to the documentation on how to add Requires-Python
2. Update twine and all tools that upload packages (and setuptools, even though it’s deprecated) to automatically transform that warning into an error overridable by --allow-no-python-requires
3. Add a backwards-compatibility API that allows you to upload packages without Requires-Python by configuring your upload endpoint to something not the default
4. Upgrade the warning to an error, dropping support for --allow-no-python-requires.

I think we could swap the order of 2 and 3 pretty easily - I’m guessing that updating the upload tools would be easier to do which is why I put them in this order, but 2 could be implemented in terms of 3.

In terms of time frame, I don’t know how aggressive you (the Warehouse team) want to be. I think the first 3 can happen as soon as an implementation is available. It’s probably a big ask to have all package maintainers switch over in a relatively short period of time, but I think a long deprecation period will be harmful given that it’s likely a large number of packages are going to start dropping support for Python 2 soon, which will probably cause a ton of headaches if people aren’t including Requires-Python in their metadata. Maybe a 6 months or so?

[Carreau]

I’m a bit worried that this proposal will result in projects setting over-strict constraints (requiring only what they test on and support, for example, when older versions may well still work on a “don’t come to us if something goes wrong” basis).

Well but if you are an advanced user and really want to get this package on an old Python version you probably can figure it out. Though if you are a new user and pip install foo and it does not work you at a loss. So I would see over-enforcing as better than being too lax.

And the all point of Python-Requires is also exactly for those old-python-users not to install a too-recent version of the package. Mostly I implemented it in Warehouse/PIP because IPython wanted to drop 2.x and couldn’t without breaking for old user. I would even expect the opposite of you, and have project to drop support ini CI, break on old python, and forget to update Python-Requires.

As far as I am aware, Python-Requires is essentially just another dependency that the resolver (new or old) has to deal with

I have not been part of some of the discussion about the resolver so pardon me if I’m mistaken. But, unlike the other dependencies, pip cannot change the current version of Python right ? Requiring Python-Requires may dramatically reduce the search space of the resolver as the Python version is not a variable. It would even be strange to me if the resolver even knows about Python-Requires, wouldn’t you just immediately filter out any package that is incompatible with the current Python and give the resulting set to the resolver ?

[edmorley]

It seems that the other half of this issue is “how can we make sure users update setuptools/twine more often?” - which would help for more than just this case.

For example:

• making twine perform version checks periodically and output a warning if it or setuptools/wheel out of date
• actively working with CI providers to ensure the tooling they use is up to date (it looks like Travis’ PyPI deploy addon already self-updates, but there are likely other common workflows that don’t use it, as well as other providers)
• updating as many of the “how to publish your Python package” docs/guides to begin with a pip install -U ...
• (plus the idea of adding support for passing back a warning message on successful upload mentioned above, so at least new installs are more likely to stay up to date in the future)

Regarding users already on older broken versions - less disruptive options (vs brownouts/blocking) might be:

• displaying a banner on the package page or when the user logs in
• emailing the user after upload