Poetry unable to find installation candidates for Nvidia c11 dependencies

• Poetry version: 1.2.0
• Python version: poetry defaults to 3.8.14 from 3.10.6
• OS version and name: macOS 13 Ventura
• pyproject.toml: see in the file below:

• I am on the latest stable Poetry version, installed using a recommended method.
• I have searched the issues of this repo and believe that this is not a duplicate.
• I have consulted the FAQ and blog for any relevant entries or release notes.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

Hi there,

Having trouble using poetry with certain NLP libraries that have Nvidia c11 related dependencies. In this case I’m using EasyNMT for machine text translation, which has a dependency on torch.

The torch related issue #4231 was recently closed, so I wanted to open a new separate issue here.

I’m unable to install the 4 Nvidia c11 dependencies:

• nvidia-cublas-cu11 (11.10.3.66)
• nvidia-cuda-nvrtc-cu11 (11.7.99)
• nvidia-cuda-runtime-cu11 (11.7.99)
• nvidia-cudnn-cu11 (8.5.0.96)

Would appreciate any advice on whether this issue is due to my setup, or an issue with poetry. Please let me know if you’re able to reproduce this issue, so I can review my environment accordingly. I have been able to lock and install these dependencies with pipenv, and then run the relevant EasyNMT code.

Thanks for the help! Alex

pyproject.toml

[tool.poetry]
name = "Translation Project"
version = "1.0"
description = "Translating text"
authors = ["me <[e-mail]>"]

[tool.poetry.dependencies]
python = "==3.8.14"
easynmt = "==2.0.2"

Checking works correctly:

poetry check

All set!

Locking works correctly:

poetry lock

The currently activated Python version 3.10.6 is not supported by the project (==3.8.14).
Trying to find and use a compatible version. 
Using python3 (3.8.14)
Creating virtualenv translation_porject-pPGuLrxo-py3.8 in /Users/alex/Library/Caches/pypoetry/virtualenvs
Updating dependencies
Resolving dependencies... Downloading https://files.pythonhosted.org/packages/ec/87/f26695307c0aa00e6938f5de795fc7f2c718a448b48d29a4c8c8dbf829d3/sentencepiece-0.1.97.tResolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz   Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  1Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  2Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  3Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  5Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  6Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  7Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  8Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  9Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  9Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  9Resolving dependencies... Downloading https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3Resolving dependencies... (84.8s)

Writing lock file

Install does not work correctly:

poetry install -vvv

The currently activated Python version 3.10.6 is not supported by the project (==3.8.14).
Trying to find and use a compatible version. 
Trying python3
Using python3 (3.8.14)
Virtualenv translation_porject-pPGuLrxo-py3.8 already exists.
Using virtualenv: /Users/alex/Library/Caches/pypoetry/virtualenvs/translation_porject-pPGuLrxo-py3.8
Project environment contains an empty path in sys_path, ignoring.
Installing dependencies from lock file

Finding the necessary packages for the current system

Package operations: 11 installs, 0 updates, 0 removals, 20 skipped

  • Installing nvidia-cublas-cu11 (11.10.3.66): Pending...
  • Installing nvidia-cuda-nvrtc-cu11 (11.7.99): Pending...
  • Installing nvidia-cuda-runtime-cu11 (11.7.99): Pending...
  • Installing nvidia-cudnn-cu11 (8.5.0.96): Pending...
[keyring.backend] Loading KWallet
[keyring.backend] Loading KWallet
[keyring.backend] Loading KWallet
[keyring.backend] Loading KWallet
[keyring.backend] Loading SecretService
[keyring.backend] Loading SecretService
[keyring.backend] Loading SecretService
[keyring.backend] Loading SecretService
[keyring.backend] Loading Windows
[keyring.backend] Loading Windows
[keyring.backend] Loading Windows
[keyring.backend] Loading Windows
[keyring.backend] Loading chainer
[keyring.backend] Loading chainer
[keyring.backend] Loading chainer
[keyring.backend] Loading chainer
[keyring.backend] Loading libsecret
[keyring.backend] Loading libsecret
[keyring.backend] Loading libsecret
[keyring.backend] Loading libsecret
[keyring.backend] Loading macOS
[keyring.backend] Loading macOS
[keyring.backend] Loading macOS
[keyring.backend] Loading macOS
Creating new session for pypi.org
Creating new session for pypi.org
Skipping wheel nvidia_cublas_cu11-11.10.3.66-py3-none-manylinux1_x86_64.whl as this is not supported by the current environment
Skipping wheel nvidia_cublas_cu11-11.10.3.66-py3-none-win_amd64.whl as this is not supported by the current environment
Skipping wheel nvidia_cuda_nvrtc_cu11-11.7.99-2-py3-none-manylinux1_x86_64.whl as this is not supported by the current environment
Skipping wheel nvidia_cuda_nvrtc_cu11-11.7.99-py3-none-manylinux1_x86_64.whl as this is not supported by the current environment
Skipping wheel nvidia_cuda_nvrtc_cu11-11.7.99-py3-none-win_amd64.whl as this is not supported by the current environment
Skipping wheel nvidia_cuda_runtime_cu11-11.7.99-py3-none-manylinux1_x86_64.whl as this is not supported by the current environment
  • Installing nvidia-cublas-cu11 (11.10.3.66): Failed

  Stack trace:

  5  /usr/local/Cellar/poetry/1.2.0/libexec/lib/python3.10/site-packages/poetry/installation/executor.py:250 in _execute_operation
        248│ 
      249│             try:
    → 250│                 result = self._do_execute_operation(operation)
      251│             except EnvCommandError as e:
      252│                 if e.e.returncode == -2:

  4  /usr/local/Cellar/poetry/1.2.0/libexec/lib/python3.10/site-packages/poetry/installation/executor.py:323 in _do_execute_operation
      321│             return 0
      322│ 
    → 323│         result: int = getattr(self, f"_execute_{method}")(operation)
      324│ 
      325│         if result != 0:

  3  /usr/local/Cellar/poetry/1.2.0/libexec/lib/python3.10/site-packages/poetry/installation/executor.py:443 in _execute_install
      441│ 
      442│     def _execute_install(self, operation: Install | Update) -> int:
    → 443│         status_code = self._install(operation)
      444│ 
      445│         self._save_url_reference(operation)

  2  /usr/local/Cellar/poetry/1.2.0/libexec/lib/python3.10/site-packages/poetry/installation/executor.py:477 in _install
      475│             archive = self._download_link(operation, Link(package.source_url))
      476│         else:
    → 477│             archive = self._download(operation)
      478│ 
      479│         operation_message = self.get_operation_message(operation)

  1  /usr/local/Cellar/poetry/1.2.0/libexec/lib/python3.10/site-packages/poetry/installation/executor.py:622 in _download
      620│ 
      621│     def _download(self, operation: Install | Update) -> Path:
    → 622│         link = self._chooser.choose_for(operation.package)
      623│ 
      624│         if link.yanked:

  RuntimeError

  Unable to find installation candidates for nvidia-cublas-cu11 (11.10.3.66)

  at /usr/local/Cellar/poetry/1.2.0/libexec/lib/python3.10/site-packages/poetry/installation/chooser.py:103 in choose_for
       99│ 
      100│             links.append(link)
      101│ 
      102│         if not links:
    → 103│             raise RuntimeError(f"Unable to find installation candidates for {package}")
      104│ 
      105│         # Get the best link
      106│         chosen = max(links, key=lambda link: self._sort_key(package, link))
      107│ 

[jspaezp]

Seems to have been reported in pytorch: https://github.com/pytorch/pytorch/issues/88049

Edit [Nov 18 2022]: For future reference https://github.com/pytorch/pytorch/pull/88826 The fix seems to be merged with torch. I will update again when the wheels with the fix land in pypi

Edit [Dec 17 2022]: 1.31.1 has landed in pypi, and it works! setting pytorch requirement to “^1.31.1”

[eric79]

I believe the torch 1.13.1 version with the fixes in it has indeed landed in PyPI: https://pypi.org/project/torch/