Warn if newly introduced upper version bounds lead poetry to install older version of a package

• I have searched the issues of this repo and believe that this is not a duplicate.
• I have searched the FAQ and general documentation and believe that my question is not already covered.

Feature Request

Many projects have upper version bounds in the form of python = "^3.8,<3.11", for example. If a user wants to install them and has an unbounded version like python = ^3.9, they usually get the helpful error message that this is not possible and that they should limit their version as well. However, some libraries used to have unbounded version requirements and then added the upper version bound later. This leads the version resolver to install outdated versions.

One reproducible example is creating a new project and having these dependencies specified:

[tool.poetry.dependencies]
python = "^3.10"
pyinstaller = "*"

This will install the old 4.5.1 version of pyinstaller, whereas the newest one is 5.6 or so. And the user does not know that anything went wrong, because there is no error message. Another example is that I tried loosening my version requirements from ❤️.11 to ❤️.12 and pandas-stubs was downgraded 3 minor versions, presumably due to the same reason.

Maybe someone can think of scenarios where this behaviour is desirable, I am not an expert, but I think in the vast majority of cases there should be at least a warning. (Or maybe this shouldn’t be done at all? Does it occur that older package versions have better new Python support than newer package versions?)

[henryiii]

Requires-python is special; the field was designed only to allow support to be dropped for a version of Python - that’s what the current solvers assume. It was not designed to allow a user to block a future updates of Python. I believe it would never have been added if that was the proposed usage. Currently pip and poetry “back solve”, that is, look through past versions of a library if this doesn’t match, trying to find an older version that does match. This is (I would argue) never correct in the “capping” case, you are never more likely to support a newer version of Python you have never seen in your past versions. The solver should either a) ignore upper caps on python-requires entirely, or b) immediately error out. (Technically, I think backsolving for any version cap is also usually incorrect, but it complicates solvers which are already complicated). https://discuss.python.org/t/requires-python-upper-limits never really got a conclusion, but that’s partially because I need to implement a feature in packaging that makes this easier to work with / detect.

Even though Poetry lists Python in the dependencies, it’s also special there too - it’s the only dependency Poetry can’t solve for (it’s fixed), and when making a lock file, it’s the only one it’s solving for a range, rather than picking a version and adding a hash. The lock file produced has a valid range (which is why adding a Python cap forces all Poetry projects that depend on that library to also include the same cap, even if they disagree with capping). The python-requires metadata is forced to be the same value as the poetry locking environment solver range, there’s only one setting. I’ve been rather hoping adding PEP 621 metadata will provide two ways to set this, and if both are set then tool.poetry setting would be the locking solver setting, and the PEP 621 one would be the metadata setting.

[neersighted]

I pretty much agree with everyone here; I think #5823 deserves particular consideration along with rethinking the aggressive encouragement of semver (and again, here it was thought that defaults are mostly harmless and will be disregarded by users who they do not work for; however, it has become clear over time that most users will not take extra steps to research and understands the specifics and we need to provide a better way for users to discover what version ranges make sense for their use case).

I also think that this is something that needs to be solved in the ecosystem and that Poetry can take steps to correct if we think we’ve taken missteps and contributed to the problem; I don’t see adding a warning that a more restrictive version range could allow a higher package range is particularly useful.

I still think that poetry update as a mechanism to discover new versions is not something we want to encourage over simple dependency audits by checking your indexes, or use of tools like Dependabot and Renovate. That being said, if the use case is revealed to be important/widespread as time goes on, I wouldn’t be opposed to making poetry update a better version discovery tool; however such an effort should be holistic and not a warning for a single edge-case.