question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Reject packages without a Requires-Python

See original GitHub issue

A problem I’ve come across many times now is that a package will drop support for an old version and they’ll even add a python_requires directive to their setup.py, only to cut a release using an old version of setuptools that doesn’t actually understand the python_requires directive and thus silently fails. See, e.g. sphinx. This causes a bunch of problems because the erroneously uploaded packages now pollute pip installs of that package unless they are removed from PyPI or a post release is made.

Since there is no good reason to upload a package without explicitly specifying a Requires-Python directive, I think we can assume that this is an error and reject packages that don’t have a Requires-Python.

My recommendation for migrating to the “always throw an error” version:

  1. Start throwing a warning when Requires-Python is missing, with a link to the documentation on how to add Requires-Python
  2. Update twine and all tools that upload packages (and setuptools, even though it’s deprecated) to automatically transform that warning into an error overridable by --allow-no-python-requires
  3. Add a backwards-compatibility API that allows you to upload packages without Requires-Python by configuring your upload endpoint to something not the default
  4. Upgrade the warning to an error, dropping support for --allow-no-python-requires.

I think we could swap the order of 2 and 3 pretty easily - I’m guessing that updating the upload tools would be easier to do which is why I put them in this order, but 2 could be implemented in terms of 3.

In terms of time frame, I don’t know how aggressive you (the Warehouse team) want to be. I think the first 3 can happen as soon as an implementation is available. It’s probably a big ask to have all package maintainers switch over in a relatively short period of time, but I think a long deprecation period will be harmful given that it’s likely a large number of packages are going to start dropping support for Python 2 soon, which will probably cause a ton of headaches if people aren’t including Requires-Python in their metadata. Maybe a 6 months or so?

Issue Analytics

  • State:open
  • Created 5 years ago
  • Reactions:8
  • Comments:15 (12 by maintainers)

github_iconTop GitHub Comments

3reactions
Carreaucommented, Apr 7, 2020

I’m a bit worried that this proposal will result in projects setting over-strict constraints (requiring only what they test on and support, for example, when older versions may well still work on a “don’t come to us if something goes wrong” basis).

Well but if you are an advanced user and really want to get this package on an old Python version you probably can figure it out. Though if you are a new user and pip install foo and it does not work you at a loss. So I would see over-enforcing as better than being too lax.

And the all point of Python-Requires is also exactly for those old-python-users not to install a too-recent version of the package. Mostly I implemented it in Warehouse/PIP because IPython wanted to drop 2.x and couldn’t without breaking for old user. I would even expect the opposite of you, and have project to drop support ini CI, break on old python, and forget to update Python-Requires.

As far as I am aware, Python-Requires is essentially just another dependency that the resolver (new or old) has to deal with

I have not been part of some of the discussion about the resolver so pardon me if I’m mistaken. But, unlike the other dependencies, pip cannot change the current version of Python right ? Requiring Python-Requires may dramatically reduce the search space of the resolver as the Python version is not a variable. It would even be strange to me if the resolver even knows about Python-Requires, wouldn’t you just immediately filter out any package that is incompatible with the current Python and give the resulting set to the resolver ?

2reactions
edmorleycommented, May 10, 2018

It seems that the other half of this issue is “how can we make sure users update setuptools/twine more often?” - which would help for more than just this case.

For example:

  • making twine perform version checks periodically and output a warning if it or setuptools/wheel out of date
  • actively working with CI providers to ensure the tooling they use is up to date (it looks like Travis’ PyPI deploy addon already self-updates, but there are likely other common workflows that don’t use it, as well as other providers)
  • updating as many of the “how to publish your Python package” docs/guides to begin with a pip install -U ...
  • (plus the idea of adding support for passing back a warning message on successful upload mentioned above, so at least new installs are more likely to stay up to date in the future)

Regarding users already on older broken versions - less disruptive options (vs brownouts/blocking) might be:

  • displaying a banner on the package page or when the user logs in
  • emailing the user after upload
Read more comments on GitHub >

github_iconTop Results From Across the Web

Reject packages without a Requires-Python · Issue #3889
A problem I've come across many times now is that a package will drop support for an old version and they'll even add...
Read more >
How to include and install local dependencies in setup.py in ...
5 Answers 5 · It seems like this wouldn't work if you haven't already installed my-package because there is no .egg until you...
Read more >
Core metadata specifications - Python Packaging User Guide
Fields defined in the following specification should be considered valid, complete and not subject to change. The required fields are: Metadata-Version. Name.
Read more >
What does Requires: Python(package) do for RHEL spec files?
It is useful if you know which module you want to use but you do not know which package contains it. While in...
Read more >
Flit Documentation
Flit is a simple way to put Python packages and modules on PyPI. ... Flit requires Python 3 and therefore needs to be...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found