Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Dependency resolution takes awfully long time

See original GitHub issue

I am on the latest Poetry version.
I have searched the issues of this repo and believe that this is not a duplicate.
If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).
OS version and name: macOS Big Sur
Poetry version: 1.1.13
Link of a Gist with the contents of your pyproject.toml file: I can provide upon request.

Issue

Version resolution in our moderately large (25 immediate dependencies) project is taking too long. For me it is about 200-250 seconds each time, and my colleagues have reported times even up to 1000 seconds.

A common scenario in this project is to update version constraint on one of the dependencies or add a new one, and then run poetry lock --no-update. It seems that the algorithm behind this command is not optimal in our case.

I will try to describe what I see in verbose (-vvv) output. Full output may be found here: https://gist.github.com/maksbotan/a72f5d801e8515b74a9f0ab7fbcef321

This concrete run had no changes in pyproject.toml and took 220 seconds anyway. As far as I see, poetry tries different solutions with different version of numpy, even if numpy version is not changed:

   0: Duplicate dependencies for numpy
   0: Different requirements found for numpy (>=1.18.5) with markers platform_machine != "aarch64" and platform_machine != "arm64" and python_version < "3.10", numpy (>=1.19.2) with markers platform_machine == "aarch64" and python_version < "3.10" and numpy (>=1.20.0) with markers platform_machine == "arm64" and python_version < "3.10".
   1: Version solving took 19.610 seconds.
   1: Tried 1 solutions.
   0: Retrying dependency resolution with the following overrides ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}}).
...
   1: Version solving took 72.396 seconds.
   1: Tried 1 solutions.
   0: Retrying dependency resolution with the following overrides ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}}).
...
   1: Version solving took 67.490 seconds.
   1: Tried 1 solutions.
   0: Retrying dependency resolution with the following overrides ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}}).
   1: Version solving took 60.766 seconds.
   1: Tried 1 solutions.
   0: Complete version solving took 220.588 seconds with 3 overrides
   0: Resolved with overrides: ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}})

I do not understand why poetry needs to retry different numpy constraints, especially when nothing changed in that regard, and why it comes up with three overrides for it with intervals that can be simplified to one. For the record, resolved version of numpy is 1.21.4.

Also, this may be related: looks like poetry tries to query global PyPI for information about packages from our private repo, even though they have [package.source] section in poetry.lock. From log:

PyPI: Getting info for humanizator (0.4.9) from PyPI
PyPI: Getting info for humanizator (0.4.9) from PyPI
PyPI: Getting info for pandorabox (0.4.21) from PyPI
PyPI: Getting info for oneq (1.11.2) from PyPI
PyPI: Getting info for pandorabox (0.4.21) from PyPI
PyPI: Getting info for humanizator (0.4.9) from PyPI
PyPI: Getting info for pandorabox (0.4.21) from PyPI
PyPI: Getting info for oneq (1.11.2) from PyPI
PyPI: Getting info for pandorabox (0.4.21) from PyPI
PyPI: Getting info for humanizator (0.4.9) from PyPI
PyPI: Getting info for pandorabox (0.4.21) from PyPI
PyPI: Getting info for oneq (1.11.2) from PyPI
PyPI: Getting info for pandorabox (0.4.21) from PyPI

Thanks in advance!