KeyAlreadyPresent error when adding atlassian-python-api

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Ubuntu 20.04.2 LTS

• Poetry version: 1.1.7

• Link of a Gist with the contents of your pyproject.toml file: https://github.com/austinbutler/poetry-keyalreadypresent/runs/3050293838

Issue

I have an existing project that was working fine with atlassian-python-api at version 3.5.2. Running poetry update resulted in:

  KeyAlreadyPresent

  Key "REPORTS" already exists.

  at /nix/store/jkj0kadrnm5bf2mral3ylkzdbrrf4mwy-python3.9-tomlkit-0.7.0/lib/python3.9/site-packages/tomlkit/container.py:176 in append
      172│                         return self
      173│                     elif current_body_element[0].is_dotted():
      174│                         raise TOMLKitError("Redefinition of an existing table")
      175│                 elif not item.is_super_table():
    → 176│                     raise KeyAlreadyPresent(key)
      177│             elif isinstance(item, AoT):
      178│                 if not isinstance(current, AoT):
      179│                     # Tried to define an AoT after a table with the same name.
      180│                     raise KeyAlreadyPresent(key)

I’m using a private PyPi repo, though, so I created a very simple repro that simply tries poetry add atlassian-python-api on a nearly empty project (linked above), and I get the same error there.

I downloaded the tar file for atlassian-python-api and if I grep for REPORTS I get one single hit from their pyproject.toml file:

/Users/abutler/Downloads/atlassian-python-api-3.12.1/pyproject.toml
171:[tool.pylint.REPORTS]

Full log from the GitHub action run:

Creating virtualenv error-repro-VVz8odaq-py3.9 in /home/runner/.cache/pypoetry/virtualenvs
Using virtualenv: /home/runner/.cache/pypoetry/virtualenvs/error-repro-VVz8odaq-py3.9
PyPI: 167 packages found for atlassian-python-api *
Using version ^3.12.1 for atlassian-python-api

Updating dependencies
Resolving dependencies...
   1: fact: error-repro is 0.1.0
   1: derived: error-repro
   1: fact: error-repro depends on atlassian-python-api (^3.12.1)
   1: fact: error-repro depends on pytest (^5.2)
   1: fact: error-repro depends on pytest (^5.2)
   1: selecting error-repro (0.1.0)
   1: derived: pytest (>=5.2,<6.0)
   1: derived: atlassian-python-api (>=3.12.1,<4.0.0)
PyPI: 15 packages found for pytest >=5.2,<6.0
PyPI: 1 packages found for atlassian-python-api >=3.12.1,<4.0.0
PyPI: Getting info for atlassian-python-api (3.12.1) from PyPI
PyPI: No dependencies found, downloading archives
PyPI: Downloading sdist: atlassian-python-api-3.12.1.tar.gz
   1: Version solving took 0.296 seconds.
   1: Tried 1 solutions.

  Stack trace:

  33  ~/.poetry/lib/poetry/_vendor/py3.9/clikit/console_application.py:131 in run
       129│             parsed_args = resolved_command.args
       130│ 
     → 131│             status_code = command.handle(parsed_args, io)
       132│         except KeyboardInterrupt:
       133│             status_code = 1

  32  ~/.poetry/lib/poetry/_vendor/py3.9/clikit/api/command/command.py:120 in handle
       118│     def handle(self, args, io):  # type: (Args, IO) -> int
       119│         try:
     → 120│             status_code = self._do_handle(args, io)
       121│         except KeyboardInterrupt:
       122│             if io.is_debug():

  31  ~/.poetry/lib/poetry/_vendor/py3.9/clikit/api/command/command.py:171 in _do_handle
       169│         handler_method = self._config.handler_method
       170│ 
     → 171│         return getattr(handler, handler_method)(args, io, self)
       172│ 
       173│     def __repr__(self):  # type: () -> str

  30  ~/.poetry/lib/poetry/_vendor/py3.9/cleo/commands/command.py:92 in wrap_handle
        90│         self._command = command
        91│ 
     →  92│         return self.handle()
        93│ 
        94│     def handle(self):  # type: () -> Optional[int]

  29  ~/.poetry/lib/poetry/console/commands/add.py:173 in handle
       171│             self._installer.whitelist([r["name"] for r in requirements])
       172│ 
     → 173│             status = self._installer.run()
       174│         except BaseException:
       175│             # Using BaseException here as some exceptions, eg: KeyboardInterrupt, do not inherit from Exception

  28  ~/.poetry/lib/poetry/installation/installer.py:103 in run
       101│         local_repo = Repository()
       102│ 
     → 103│         return self._do_install(local_repo)
       104│ 
       105│     def dry_run(self, dry_run=True):  # type: (bool) -> Installer

  27  ~/.poetry/lib/poetry/installation/installer.py:235 in _do_install
       233│             )
       234│ 
     → 235│             ops = solver.solve(use_latest=self._whitelist)
       236│         else:
       237│             self._io.write_line("Installing dependencies from lock file")

  26  ~/.poetry/lib/poetry/puzzle/solver.py:65 in solve
        63│         with self._provider.progress():
        64│             start = time.time()
     →  65│             packages, depths = self._solve(use_latest=use_latest)
        66│             end = time.time()
        67│ 

  25  ~/.poetry/lib/poetry/puzzle/solver.py:233 in _solve
       231│ 
       232│         try:
     → 233│             result = resolve_version(
       234│                 self._package, self._provider, locked=locked, use_latest=use_latest
       235│             )

  24  ~/.poetry/lib/poetry/mixology/__init__.py:7 in resolve_version
       5│     solver = VersionSolver(root, provider, locked=locked, use_latest=use_latest)
       6│ 
     → 7│     return solver.solve()
       8│ 

  23  ~/.poetry/lib/poetry/mixology/version_solver.py:84 in solve
        82│             while next is not None:
        83│                 self._propagate(next)
     →  84│                 next = self._choose_package_version()
        85│ 
        86│             return self._result()

  22  ~/.poetry/lib/poetry/mixology/version_solver.py:400 in _choose_package_version
       398│             version = locked
       399│ 
     → 400│         version = self._provider.complete_package(version)
       401│ 
       402│         conflict = False

  21  ~/.poetry/lib/poetry/puzzle/provider.py:433 in complete_package
       431│             package = DependencyPackage(
       432│                 package.dependency,
     → 433│                 self._pool.package(
       434│                     package.name,
       435│                     package.version.text,

  20  ~/.poetry/lib/poetry/repositories/pool.py:135 in package
       133│             for idx, repo in enumerate(self._repositories):
       134│                 try:
     → 135│                     package = repo.package(name, version, extras=extras)
       136│                 except PackageNotFound:
       137│                     continue

  19  ~/.poetry/lib/poetry/repositories/pypi_repository.py:158 in package
       156│         extras=None,  # type: (Union[list, None])
       157│     ):  # type: (...) -> Package
     → 158│         return self.get_release_info(name, version).to_package(name=name, extras=extras)
       159│ 
       160│     def search(self, query):

  18  ~/.poetry/lib/poetry/repositories/pypi_repository.py:223 in get_release_info
       221│             return PackageInfo.load(self._get_release_info(name, version))
       222│ 
     → 223│         cached = self._cache.remember_forever(
       224│             "{}:{}".format(name, version), lambda: self._get_release_info(name, version)
       225│         )

  17  ~/.poetry/lib/poetry/_vendor/py3.9/cachy/repository.py:174 in remember_forever
       172│             return val
       173│ 
     → 174│         val = value(callback)
       175│ 
       176│         self.forever(key, val)

  16  ~/.poetry/lib/poetry/_vendor/py3.9/cachy/helpers.py:6 in value
       4│ def value(val):
       5│     if callable(val):
     → 6│         return val()
       7│ 
       8│     return val

  15  ~/.poetry/lib/poetry/repositories/pypi_repository.py:224 in <lambda>
       222│ 
       223│         cached = self._cache.remember_forever(
     → 224│             "{}:{}".format(name, version), lambda: self._get_release_info(name, version)
       225│         )
       226│ 

  14  ~/.poetry/lib/poetry/repositories/pypi_repository.py:304 in _get_release_info
       302│                 return data.asdict()
       303│ 
     → 304│             info = self._get_info_from_urls(urls)
       305│ 
       306│             data.requires_dist = info.requires_dist

  13  ~/.poetry/lib/poetry/repositories/pypi_repository.py:419 in _get_info_from_urls
       417│                 return self._get_info_from_wheel(platform_specific_wheels[0])
       418│ 
     → 419│         return self._get_info_from_sdist(urls["sdist"][0])
       420│ 
       421│     def _get_info_from_wheel(self, url):  # type: (str) -> PackageInfo

  12  ~/.poetry/lib/poetry/repositories/pypi_repository.py:447 in _get_info_from_sdist
       445│             self._download(url, str(filepath))
       446│ 
     → 447│             return PackageInfo.from_sdist(filepath)
       448│ 
       449│     def _download(self, url, dest):  # type: (str, str) -> None

  11  ~/.poetry/lib/poetry/inspection/info.py:561 in from_sdist
       559│         """
       560│         if path.is_file():
     → 561│             return cls._from_sdist_file(path=path)
       562│ 
       563│         # if we get here then it is neither an sdist instance nor a file

  10  ~/.poetry/lib/poetry/inspection/info.py:291 in _from_sdist_file
       289│ 
       290│             # now this is an unpacked directory we know how to deal with
     → 291│             new_info = cls.from_directory(path=sdist_dir)
       292│ 
       293│         if not info:

   9  ~/.poetry/lib/poetry/inspection/info.py:530 in from_directory
       528│             order to gather metadata.
       529│         """
     → 530│         project_package = cls._get_poetry_package(path)
       531│         if project_package:
       532│             info = cls.from_package(project_package)

   8  ~/.poetry/lib/poetry/inspection/info.py:439 in _get_poetry_package
       437│         # Note: we ignore any setup.py file at this step
       438│         # TODO: add support for handling non-poetry PEP-517 builds
     → 439│         if PyProjectTOML(path.joinpath("pyproject.toml")).is_poetry_project():
       440│             return Factory().create_poetry(path).package
       441│ 

   7  ~/.poetry/lib/poetry/_vendor/py3.9/poetry/core/pyproject/toml.py:64 in is_poetry_project
       62│         if self.file.exists():
       63│             try:
     → 64│                 _ = self.poetry_config
       65│                 return True
       66│             except PyProjectException:

   6  ~/.poetry/lib/poetry/_vendor/py3.9/poetry/core/pyproject/toml.py:54 in poetry_config
       52│     def poetry_config(self):  # type: () -> Optional[TOMLDocument]
       53│         if self._poetry_config is None:
     → 54│             self._poetry_config = self.data.get("tool", {}).get("poetry")
       55│             if self._poetry_config is None:
       56│                 raise PyProjectException(

   5  /opt/hostedtoolcache/Python/3.9.5/x64/lib/python3.9/_collections_abc.py:762 in get
        760│         'D.get(k[,d]) -> D[k] if k in D, else d.  d defaults to None.'
        761│         try:
     →  762│             return self[key]
        763│         except KeyError:
        764│             return default

   4  ~/.poetry/lib/poetry/_vendor/py3.9/tomlkit/container.py:559 in __getitem__
       557│             # so we need a proxy to retrieve the proper objects
       558│             # from the parent container
     → 559│             return OutOfOrderTableProxy(self, idx)
       560│ 
       561│         item = self._body[idx][1]

   3  ~/.poetry/lib/poetry/_vendor/py3.9/tomlkit/container.py:699 in __init__
       697│                 table_idx = len(self._tables) - 1
       698│                 for k, v in item.value.body:
     → 699│                     self._internal_container.append(k, v)
       700│                     self._tables_map[k] = table_idx
       701│                     if k is not None:

   2  ~/.poetry/lib/poetry/_vendor/py3.9/tomlkit/container.py:169 in append
       167│ 
       168│                         for k, v in item.value.body:
     → 169│                             current.append(k, v)
       170│ 
       171│                         return self

   1  ~/.poetry/lib/poetry/_vendor/py3.9/tomlkit/items.py:922 in append
        920│             _item = item(_item)
        921│ 
     →  922│         self._value.append(key, _item)
        923│ 
        924│         if isinstance(key, Key):

  KeyAlreadyPresent

  Key "REPORTS" already exists.

  at ~/.poetry/lib/poetry/_vendor/py3.9/tomlkit/container.py:175 in append
      171│                         return self
      172│                     elif current_body_element[0].is_dotted():
      173│                         raise TOMLKitError("Redefinition of an existing table")
      174│                 elif not item.is_super_table():
    → 175│                     raise KeyAlreadyPresent(key)
      176│             elif isinstance(item, AoT):
      177│                 if not isinstance(current, AoT):
      178│                     # Tried to define an AoT after a table with the same name.
      179│                     raise KeyAlreadyPresent(key)

[austinbutler]

You can install from my fork to get around this for now.

[tool.poetry.dependencies]
 atlassian-python-api = { git = "https://github.com/austinbutler/atlassian-python-api.git", branch = "7474d6ca08b10552ed1e7655adaaff01b093facb" }

Note though that I branched from master, which has some commits not yet released.

[austinbutler]

Seems like TOML Kit is tripped up by ['tool.pylint.MESSAGES CONTROL'] in the pyproject.toml. If I change that to [tool.pylint.'MESSAGES CONTROL'] there’s no longer a duplicate table name. Opened a PR atlassian-python-api to “fix” that (not intimately familiar with TOML so not entirely sure whether it’s actually invalid). Also created an issue with TOML Kit. Since Poetry is not at fault this can probably be closed. I could close it myself, but not sure if it’s handy to have open in case others end up with the same issue and come here.