Update not working with two Artifactory sources

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: macOS 10.15.7

• Poetry version: 1.1.2

• Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/austinbutler/aa1c53dc846eb9a3e913d0444c256b82

Issue

For installing packages from a private Artifactory, the following works:

[[tool.poetry.source]]
name = "artifactory"
url = "https://artifacts.example.com/artifactory/api/pypi/pypi/simple/"
default = true

But when trying to publish to that source it does not work. So I defined a second source that works for publishing:

[[tool.poetry.source]]
name = "artifactory-upload"
url = "https://artifacts.example.com/artifactory/api/pypi/pypi/"

Then I can publish with poetry publish --build --repository artifactory-upload.

But once that second source is added I get this from poetry update:

Using virtualenv: /Users/abutler/Documents/my-package/.venv
Updating dependencies
Resolving dependencies...
   1: fact: my-package is 1.2.0
   1: derived: my-package
   1: fact: my-package depends on requests (^2.24.0)
   1: fact: my-package depends on jpype1 (^1.0.2)
   1: fact: my-package depends on black (^20.8b1)
   1: fact: my-package depends on isort (^5.5.4)
   1: fact: my-package depends on mypy (^0.782)
   1: fact: my-package depends on pydocstyle (^5.1.1)
   1: fact: my-package depends on pylint (^2.6.0)
   1: fact: my-package depends on pytest (^6.1.1)
   1: fact: my-package depends on black (^20.8b1)
   1: fact: my-package depends on isort (^5.5.4)
   1: fact: my-package depends on mypy (^0.782)
   1: fact: my-package depends on pydocstyle (^5.1.1)
   1: fact: my-package depends on pylint (^2.6.0)
   1: fact: my-package depends on pytest (^6.1.1)
   1: selecting my-package (1.2.0)
   1: derived: pytest (^6.1.1)
   1: derived: pylint (^2.6.0)
   1: derived: pydocstyle (^5.1.1)
   1: derived: mypy (^0.782)
   1: derived: isort (^5.5.4)
   1: derived: black (^20.8b1)
   1: derived: jpype1 (^1.0.2)
   1: derived: requests (^2.24.0)
artifactory: 1 packages found for pytest >=6.1.1,<7.0.0
   1: fact: pytest (6.1.1) depends on attrs (>=17.4.0)
   1: fact: pytest (6.1.1) depends on iniconfig (*)
   1: fact: pytest (6.1.1) depends on packaging (*)
   1: fact: pytest (6.1.1) depends on pluggy (>=0.12,<1.0)
   1: fact: pytest (6.1.1) depends on py (>=1.8.2)
   1: fact: pytest (6.1.1) depends on toml (*)
   1: fact: pytest (6.1.1) depends on atomicwrites (>=1.0)
   1: fact: pytest (6.1.1) depends on colorama (*)
   1: selecting pytest (6.1.1)
   1: derived: colorama (*)
   1: derived: atomicwrites (>=1.0)
   1: derived: toml (*)
   1: derived: py (>=1.8.2)
   1: derived: pluggy (>=0.12,<1.0)
   1: derived: packaging (*)
   1: derived: iniconfig (*)
   1: derived: attrs (>=17.4.0)
artifactory: 1 packages found for pylint >=2.6.0,<3.0.0
   1: fact: pylint (2.6.0) depends on astroid (>=2.4.0,<=2.5)
   1: fact: pylint (2.6.0) depends on isort (>=4.2.5,<6)
   1: fact: pylint (2.6.0) depends on mccabe (>=0.6,<0.7)
   1: fact: pylint (2.6.0) depends on toml (>=0.7.1)
   1: fact: pylint (2.6.0) depends on colorama (*)
   1: selecting pylint (2.6.0)
   1: derived: toml (>=0.7.1)
   1: derived: mccabe (>=0.6,<0.7)
   1: derived: astroid (>=2.4.0,<=2.5)
artifactory: 1 packages found for pydocstyle >=5.1.1,<6.0.0
   1: fact: pydocstyle (5.1.1) depends on snowballstemmer (*)
   1: selecting pydocstyle (5.1.1)
   1: derived: snowballstemmer (*)
artifactory: 1 packages found for mypy >=0.782,<0.783
   1: fact: mypy (0.782) depends on typed-ast (>=1.4.0,<1.5.0)
   1: fact: mypy (0.782) depends on typing-extensions (>=3.7.4)
   1: fact: mypy (0.782) depends on mypy-extensions (>=0.4.3,<0.5.0)
   1: selecting mypy (0.782)
   1: derived: mypy-extensions (>=0.4.3,<0.5.0)
   1: derived: typing-extensions (>=3.7.4)
   1: derived: typed-ast (>=1.4.0,<1.5.0)
artifactory: 4 packages found for isort >=5.5.4,<6.0.0
   1: selecting isort (5.6.1)
artifactory: 1 packages found for black >=20.8b1,<21.0
   1: fact: black (20.8b1) depends on click (>=7.1.2)
   1: fact: black (20.8b1) depends on appdirs (*)
   1: fact: black (20.8b1) depends on toml (>=0.10.1)
   1: fact: black (20.8b1) depends on typed-ast (>=1.4.0)
   1: fact: black (20.8b1) depends on regex (>=2020.1.8)
   1: fact: black (20.8b1) depends on pathspec (>=0.6,<1)
   1: fact: black (20.8b1) depends on typing_extensions (>=3.7.4)
   1: fact: black (20.8b1) depends on mypy_extensions (>=0.4.3)
   1: selecting black (20.8b1)
   1: derived: pathspec (>=0.6,<1)
   1: derived: regex (>=2020.1.8)
   1: derived: toml (>=0.10.1)
   1: derived: appdirs (*)
   1: derived: click (>=7.1.2)
artifactory: 1 packages found for jpype1 >=1.0.2,<2.0.0
   1: selecting jpype1 (1.0.2)
artifactory: 1 packages found for requests >=2.24.0,<3.0.0
   1: fact: requests (2.24.0) depends on chardet (>=3.0.2,<4)
   1: fact: requests (2.24.0) depends on idna (>=2.5,<3)
   1: fact: requests (2.24.0) depends on urllib3 (>=1.21.1,<1.25.0 || >1.25.0,<1.25.1 || >1.25.1,<1.26)
   1: fact: requests (2.24.0) depends on certifi (>=2017.4.17)
   1: selecting requests (2.24.0)
   1: derived: certifi (>=2017.4.17)
   1: derived: urllib3 (>=1.21.1,<1.25.0 || >1.25.0,<1.25.1 || >1.25.1,<1.26)
   1: derived: idna (>=2.5,<3)
   1: derived: chardet (>=3.0.2,<4)
artifactory: 41 packages found for colorama *
   1: selecting colorama (0.4.3)
artifactory: 7 packages found for atomicwrites >=1.0
   1: selecting atomicwrites (1.4.0)
artifactory: 1 packages found for toml >=0.10.1
   1: selecting toml (0.10.1)
artifactory: 2 packages found for py >=1.8.2
   1: selecting py (1.9.0)
artifactory: 3 packages found for pluggy >=0.12,<1.0
   1: selecting pluggy (0.13.1)
artifactory: 30 packages found for packaging *
   1: fact: packaging (20.4) depends on pyparsing (>=2.0.2)
   1: fact: packaging (20.4) depends on six (*)
   1: selecting packaging (20.4)
   1: derived: six (*)
   1: derived: pyparsing (>=2.0.2)
artifactory: 3 packages found for iniconfig *
   1: selecting iniconfig (1.0.1)
artifactory: 8 packages found for attrs >=17.4.0
   1: selecting attrs (20.2.0)
artifactory: 2 packages found for mccabe >=0.6,<0.7
   1: selecting mccabe (0.6.1)
artifactory: 3 packages found for astroid >=2.4.0,<=2.5
   1: fact: astroid (2.4.2) depends on lazy-object-proxy (>=1.4.0,<1.5.0)
   1: fact: astroid (2.4.2) depends on six (>=1.12,<2.0)
   1: fact: astroid (2.4.2) depends on wrapt (>=1.11,<2.0)
   1: selecting astroid (2.4.2)
   1: derived: wrapt (>=1.11,<2.0)
   1: derived: six (>=1.12,<2.0)
   1: derived: lazy-object-proxy (>=1.4.0,<1.5.0)
artifactory: 8 packages found for snowballstemmer *
   1: Version solving took 7.217 seconds.
   1: Tried 1 solutions.

  Stack trace:

  15  .venv/lib/python3.8/site-packages/clikit/console_application.py:131 in run
       129│             parsed_args = resolved_command.args
       130│
     → 131│             status_code = command.handle(parsed_args, io)
       132│         except KeyboardInterrupt:
       133│             status_code = 1

  14  .venv/lib/python3.8/site-packages/clikit/api/command/command.py:120 in handle
       118│     def handle(self, args, io):  # type: (Args, IO) -> int
       119│         try:
     → 120│             status_code = self._do_handle(args, io)
       121│         except KeyboardInterrupt:
       122│             if io.is_debug():

  13  .venv/lib/python3.8/site-packages/clikit/api/command/command.py:171 in _do_handle
       169│         handler_method = self._config.handler_method
       170│
     → 171│         return getattr(handler, handler_method)(args, io, self)
       172│
       173│     def __repr__(self):  # type: () -> str

  12  .venv/lib/python3.8/site-packages/cleo/commands/command.py:92 in wrap_handle
        90│         self._command = command
        91│
     →  92│         return self.handle()
        93│
        94│     def handle(self):  # type: () -> Optional[int]

  11  .venv/lib/python3.8/site-packages/poetry/console/commands/update.py:47 in handle
       45│         self._installer.update(True)
       46│
     → 47│         return self._installer.run()
       48│

  10  .venv/lib/python3.8/site-packages/poetry/installation/installer.py:103 in run
       101│         local_repo = Repository()
       102│
     → 103│         return self._do_install(local_repo)
       104│
       105│     def dry_run(self, dry_run=True):  # type: (bool) -> Installer

   9  .venv/lib/python3.8/site-packages/poetry/installation/installer.py:235 in _do_install
       233│             )
       234│
     → 235│             ops = solver.solve(use_latest=self._whitelist)
       236│         else:
       237│             self._io.write_line("Installing dependencies from lock file")

   8  .venv/lib/python3.8/site-packages/poetry/puzzle/solver.py:65 in solve
        63│         with self._provider.progress():
        64│             start = time.time()
     →  65│             packages, depths = self._solve(use_latest=use_latest)
        66│             end = time.time()
        67│

   7  .venv/lib/python3.8/site-packages/poetry/puzzle/solver.py:233 in _solve
       231│
       232│         try:
     → 233│             result = resolve_version(
       234│                 self._package, self._provider, locked=locked, use_latest=use_latest
       235│             )

   6  .venv/lib/python3.8/site-packages/poetry/mixology/__init__.py:7 in resolve_version
       5│     solver = VersionSolver(root, provider, locked=locked, use_latest=use_latest)
       6│
     → 7│     return solver.solve()
       8│

   5  .venv/lib/python3.8/site-packages/poetry/mixology/version_solver.py:84 in solve
        82│             while next is not None:
        83│                 self._propagate(next)
     →  84│                 next = self._choose_package_version()
        85│
        86│             return self._result()

   4  .venv/lib/python3.8/site-packages/poetry/mixology/version_solver.py:374 in _choose_package_version
       372│         if locked is None or not dependency.constraint.allows(locked.version):
       373│             try:
     → 374│                 packages = self._provider.search_for(dependency)
       375│             except ValueError as e:
       376│                 self._add_incompatibility(

   3  .venv/lib/python3.8/site-packages/poetry/puzzle/provider.py:139 in search_for
       137│             packages = self.search_for_url(dependency)
       138│         else:
     → 139│             packages = self._pool.find_packages(dependency)
       140│
       141│             packages.sort(

   2  .venv/lib/python3.8/site-packages/poetry/repositories/pool.py:165 in find_packages
       163│         packages = []
       164│         for repo in self._repositories:
     → 165│             packages += repo.find_packages(dependency)
       166│
       167│         return packages

   1  .venv/lib/python3.8/site-packages/poetry/repositories/legacy_repository.py:261 in find_packages
       259│             versions = self._cache.store("matches").get(key)
       260│         else:
     → 261│             page = self._get("/{}/".format(dependency.name.replace(".", "-")))
       262│             if page is None:
       263│                 return []

  RepositoryError

  500 Server Error: Internal Server Error for url: https://artifacts.example.com/artifactory/api/pypi/pypi/snowballstemmer/

  at .venv/lib/python3.8/site-packages/poetry/repositories/legacy_repository.py:390 in _get
      386│             if response.status_code == 404:
      387│                 return
      388│             response.raise_for_status()
      389│         except requests.HTTPError as e:
    → 390│             raise RepositoryError(e)
      391│
      392│         if response.status_code in (401, 403):
      393│             self._log(
      394│                 "Authorization error accessing {url}".format(url=url), level="warn"

If I simply remove the artifactory-upload source from my pyproject.toml poetry update works fine. What’s really confusing is at least according to the debug output it’s trying to use the primary artifactory source, but then the url in the exception appears to be the artifactory-upload source. Maybe that’s the bug, using the wrong repo URL.

[pawamoy]

We could also get rid of the url in pyproject.toml when/since it’s already configured in Poetry’s global config.

[abn]

@austinbutler The new behaviour is undocumented. The source gets merged into available repositories to allow them to use configured credentials. We should probably add a flag publish = false to these repositories when injected into the runtime configuration so that the publish logic ignores them.