Poetry not respecting order of evaluation for repository sources

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• OS version and name: Mac OS 10.15.3

• Poetry version: 1.0.5

Issue

So, I am trying to configure a pyproject.toml file that supports pulling files from pypi.org whenever possible (ie: because on my site the performance is better) but will support pulling packages that aren’t found on pypi.org from a secondary location (ie: a private pypi repo). So to start with I added a section like what follows to a sample toml file for testing:

[[tool.poetry.source]]
name = 'default'
url = 'https://pypi.python.org/simple'
default = true

[[tool.poetry.source]]
name = "private"
url = "https://private/server/url
secondary = true

[tool.poetry.dependencies]
python = "^3.6"
sphinx = "*"

Based on the docs, I was under the impression that by putting default=true on the first repo config, and secondary=true on the other one, that my goal would be achieved: if a package exists on pypi.org it’d pull it from there and if not it’d pull it from my secondary repo. However, that does not seem to be the case.

For the sake of this discussion I am using average performance metrics from running poetry lock on this toml file and comparing the length of time it takes to resolve the dependencies of the one Python package mentioned (ie: sphinx which is a standard Python package available on pypi.org).

So, running poetry lock on this file using the configuration I posted above takes about 20-25 seconds to complete. For comparison purposes I simply removed the second source definition from the toml file giving me:

[[tool.poetry.source]]
name = 'default'
url = 'https://pypi.python.org/simple'
default = true

[tool.poetry.dependencies]
python = "^3.6"
sphinx = "*"

Re-running the same common against this toml file takes between 2 and 3 seconds to complete - a full order of magnitude difference in performance. Now, for my 3rd and final test I removed the first repo source leaving just the second one, giving me the following configuration block:

[[tool.poetry.source]]
name = "private"
url = "https://private/server/url
secondary = true

[tool.poetry.dependencies]
python = "^3.6"
sphinx = "*"

Re-running the same command again takes about the same 20-25 seconds to complete (NOTE: our private pypi repository is also a mirror of the public pypi.org repo so I can compare the same package versions and resolution times in all cases).

So, based on these results, the only explanation I can see is that when the second repository definition is in place the public pypi.org repo is being ignored or something and the secondary / private repo is still being accessed for some reason.

My expectation here is that poetry would iterate over the various repos defined in the toml file in order of declaration. For each package listed in the toml file it should then try to access / index each package against each repository stopping when it finds the first match. If this were true then I would have expected my first test case above to perform identically to the second one because the package I’m testing (sphinx) should exist on pypi.org, and all of it’s transitive dependencies will be available there as well. However this is obviously not the case.

So my question is - is this a bug or is there some additional configuration options I need to specify to get this to work as I was expecting?

[alimantu]

I face same issue here, it seems to be, that poetry/pip checks packages for both default and secondary sources. It makes sense if we are talking about robustness - you’ll always get version that will fit you best from poetry/pip point of view, but even so I would suggest at least some key/argument to avoid this deep check in case package fit the dependencies already found during search in some previous repo.

[kakarukeys]

I am seeing either a regression of this bug, or a missing instruction in the documentation.

I followed this Install dependencies from a private repository to add a private pypi repo as a source for package installation, setting secondary = true. The lock file generated put the private pypi as source for all packages, public and private.

Adding an extra section tool.poetry.source with the official PyPI mitigated the issue, but this is not documented. The official PyPI should be an implicit source that does not require declaration.