Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

VCS dependency with extras requires explicit branch (due to double clone)

See original GitHub issue

Poetry version: 1.2.2
Python version: 3.10.8
OS version and name: Arch Linux, Windows 10/11, macOS 13

I am on the latest stable Poetry version, installed using a recommended method.
I have searched the issues of this repo and believe that this is not a duplicate.
I have consulted the FAQ and blog for any relevant entries or release notes.
If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

When using a VCS dependency with extras and the Dulwich Git client, an explicit branch must be provided. The following pyproject.toml will fail to clone:

[tool.poetry]
name = "extra-second-clone"
version = "0.1.0"
description = ""
authors = ["Bjorn Neergaard <bjorn@neersighted.com>"]
readme = "README.md"
packages = [{include = "extra_second_clone"}]

[tool.poetry.dependencies]
python = "^3.10"
poetry-core = {git = "https://github.com/python-poetry/poetry-core", extras = ["demo"]}


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

$ poetry lock -vvv
Loading configuration file /home/neersighted/.config/pypoetry/config.toml
Loading configuration file /home/neersighted/.config/pypoetry/auth.toml
Trying to detect current active python executable as specified in the config.
Found: /home/neersighted/.local/share/pyenv/versions/3.11.0/bin/python
Virtualenv extra-second-clone-IgA_y9qo-py3.11 already exists.
Using virtualenv: /home/neersighted/.cache/pypoetry/virtualenvs/extra-second-clone-IgA_y9qo-py3.11
Project environment contains an empty path in sys_path, ignoring.
Updating dependencies
Resolving dependencies...
   1: fact: extra-second-clone is 0.1.0
   1: derived: extra-second-clone
[keyring.backend] Loading KWallet
[keyring.backend] Loading SecretService
[keyring.backend] Loading Windows
[keyring.backend] Loading chainer
[keyring.backend] Loading libsecret
[keyring.backend] Loading macOS
No suitable keyring backend found
No suitable keyring backends were found
Keyring is not available, credentials will be stored and retrieved from configuration files as plaintext.
[urllib3.connectionpool] Starting new HTTPS connection (1): github.com:443
[urllib3.connectionpool] https://github.com:443 "GET /python-poetry/poetry-core/info/refs?service=git-upload-pack HTTP/1.1" 200 None
Cloning https://github.com/python-poetry/poetry-core at 'HEAD' to /home/neersighted/.cache/pypoetry/virtualenvs/extra-second-clone-IgA_y9qo-py3.11/src/poetry-core
   1: fact: extra-second-clone depends on poetry-core[demo] (1.3.2)
   1: selecting extra-second-clone (0.1.0)
   1: derived: poetry-core[demo] (1.3.2) @ git+https://github.com/python-poetry/poetry-core
   1: fact: poetry-core[demo] (1.3.2) depends on poetry-core (1.3.2)
   1: selecting poetry-core[demo] (1.3.2 b0b1823)
   1: derived: poetry-core (1.3.2) @ git+https://github.com/python-poetry/poetry-core@HEAD
[urllib3.connectionpool] Starting new HTTPS connection (1): github.com:443
[urllib3.connectionpool] https://github.com:443 "GET /python-poetry/poetry-core/info/refs?service=git-upload-pack HTTP/1.1" 200 None
Cloning https://github.com/python-poetry/poetry-core at 'HEAD' to /home/neersighted/.cache/pypoetry/virtualenvs/extra-second-clone-IgA_y9qo-py3.11/src/poetry-core
   1: Version solving took 1.310 seconds.
   1: Tried 1 solutions.

Failed to clone https://github.com/python-poetry/poetry-core at 'HEAD', verify ref exists on remote.

The following pyproject.toml will succeed:

[tool.poetry]
name = "extra-second-clone"
version = "0.1.0"
description = ""
authors = ["Bjorn Neergaard <bjorn@neersighted.com>"]
readme = "README.md"
packages = [{include = "extra_second_clone"}]

[tool.poetry.dependencies]
python = "^3.10"
poetry-core = {git = "https://github.com/python-poetry/poetry-core", extras = ["demo"]}


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

$ poetry lock -vvv
Loading configuration file /home/neersighted/.config/pypoetry/config.toml
Loading configuration file /home/neersighted/.config/pypoetry/auth.toml
Trying to detect current active python executable as specified in the config.
Found: /home/neersighted/.local/share/pyenv/versions/3.11.0/bin/python
Virtualenv extra-second-clone-IgA_y9qo-py3.11 already exists.
Using virtualenv: /home/neersighted/.cache/pypoetry/virtualenvs/extra-second-clone-IgA_y9qo-py3.11
Project environment contains an empty path in sys_path, ignoring.
Updating dependencies
Resolving dependencies...
   1: fact: extra-second-clone is 0.1.0
   1: derived: extra-second-clone
[keyring.backend] Loading KWallet
[keyring.backend] Loading SecretService
[keyring.backend] Loading Windows
[keyring.backend] Loading chainer
[keyring.backend] Loading libsecret
[keyring.backend] Loading macOS
No suitable keyring backend found
No suitable keyring backends were found
Keyring is not available, credentials will be stored and retrieved from configuration files as plaintext.
[urllib3.connectionpool] Starting new HTTPS connection (1): github.com:443
[urllib3.connectionpool] https://github.com:443 "GET /python-poetry/poetry-core/info/refs?service=git-upload-pack HTTP/1.1" 200 None
Cloning https://github.com/python-poetry/poetry-core at 'main' to /home/neersighted/.cache/pypoetry/virtualenvs/extra-second-clone-IgA_y9qo-py3.11/src/poetry-core
   1: fact: extra-second-clone depends on poetry-core[demo] (1.3.2)
   1: selecting extra-second-clone (0.1.0)
   1: derived: poetry-core[demo] (1.3.2) @ git+https://github.com/python-poetry/poetry-core@main
   1: fact: poetry-core[demo] (1.3.2) depends on poetry-core (1.3.2)
   1: selecting poetry-core[demo] (1.3.2 b0b1823)
   1: derived: poetry-core (1.3.2) @ git+https://github.com/python-poetry/poetry-core@main
[urllib3.connectionpool] Starting new HTTPS connection (1): github.com:443
[urllib3.connectionpool] https://github.com:443 "GET /python-poetry/poetry-core/info/refs?service=git-upload-pack HTTP/1.1" 200 None
Cloning https://github.com/python-poetry/poetry-core at 'main' to /home/neersighted/.cache/pypoetry/virtualenvs/extra-second-clone-IgA_y9qo-py3.11/src/poetry-core
   1: selecting poetry-core (1.3.2 b0b1823)
   1: Version solving took 1.513 seconds.
   1: Tried 1 solutions.

Writing lock file

Finally, dropping the use of extras will also succeed:

[tool.poetry]
name = "extra-second-clone"
version = "0.1.0"
description = ""
authors = ["Bjorn Neergaard <bjorn@neersighted.com>"]
readme = "README.md"
packages = [{include = "extra_second_clone"}]

[tool.poetry.dependencies]
python = "^3.10"
poetry-core = {git = "https://github.com/python-poetry/poetry-core"}


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

$ poetry lock -vvv
Loading configuration file /home/neersighted/.config/pypoetry/config.toml
Loading configuration file /home/neersighted/.config/pypoetry/auth.toml
Trying to detect current active python executable as specified in the config.
Found: /home/neersighted/.local/share/pyenv/versions/3.11.0/bin/python
Virtualenv extra-second-clone-IgA_y9qo-py3.11 already exists.
Using virtualenv: /home/neersighted/.cache/pypoetry/virtualenvs/extra-second-clone-IgA_y9qo-py3.11
Project environment contains an empty path in sys_path, ignoring.
Updating dependencies
Resolving dependencies...
   1: fact: extra-second-clone is 0.1.0
   1: derived: extra-second-clone
[keyring.backend] Loading KWallet
[keyring.backend] Loading SecretService
[keyring.backend] Loading Windows
[keyring.backend] Loading chainer
[keyring.backend] Loading libsecret
[keyring.backend] Loading macOS
No suitable keyring backend found
No suitable keyring backends were found
Keyring is not available, credentials will be stored and retrieved from configuration files as plaintext.
[urllib3.connectionpool] Starting new HTTPS connection (1): github.com:443
[urllib3.connectionpool] https://github.com:443 "GET /python-poetry/poetry-core/info/refs?service=git-upload-pack HTTP/1.1" 200 None
Cloning https://github.com/python-poetry/poetry-core at 'HEAD' to /home/neersighted/.cache/pypoetry/virtualenvs/extra-second-clone-IgA_y9qo-py3.11/src/poetry-core
   1: fact: extra-second-clone depends on poetry-core (1.3.2)
   1: selecting extra-second-clone (0.1.0)
   1: derived: poetry-core (1.3.2) @ git+https://github.com/python-poetry/poetry-core
   1: selecting poetry-core (1.3.2 b0b1823)
   1: Version solving took 0.945 seconds.
   1: Tried 1 solutions.

Writing lock file

The keen-eyed reader will notice that the failures occur during a second clone (?!); when extras are present, the ref HEAD cannot be resolved by Dulwich during the second clone. The second clone seems to be a bug, and eliminating it should solve the failure; however, identifying the reason the second clone with the same ref fails would be preferred so we can better understand what is going wrong.

Issue Analytics

State:
Created 10 months ago
Comments:5 (4 by maintainers)

Top GitHub Comments

1reaction

neersightedcommented, Nov 13, 2022

PTAL @radoering @dimbleby @abn

0reactions

robtaylorcommented, Dec 6, 2022

I just hit the same issue…

Top Results From Across the Web

How to state in requirements.txt a direct github source

I forked a project where the package name that you pip install has a dash in it ( package-two ), but the module...

Build customization - Read the Docs

Unshallow clone. Cancel build based on a condition. Generate documentation from annotated sources with Doxygen. Use MkDocs extensions with extra required ...

Caching Dependencies - CircleCI

CircleCI manual dependency caching requires you to be explicit about what you cache and how you cache it. See the save cache section...

Repositories - Composer

A repository is a package source. It's a list of packages/versions. Composer will look in all your repositories to find the packages your...

History | Poetry - Python dependency management and ...

Fix an issue where the source field of a dependency with extras was ignored ... Fix an issue where invalid PEP 508 requirements...