pytorch and poetry

• [X ] I am on the latest Poetry version.
• [X ] I have searched the issues of this repo and believe that this is not a duplicate.
• [ X] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: docker on linux with cuda
• Poetry version: 1.1.7
• Link of a Gist with the contents of your pyproject.toml file: <below

Issue

pytorch is now pep503 compliant (https://github.com/pytorch/pytorch/issues/25639#issuecomment-861707149) but I still can’t add torch and torchvision. I’ve tried using this:

torch = {url = "https://download.pytorch.org/whl/cu111/torch-1.9.0%2Bcu111-cp38-cp38-linux_x86_64.whl"}
torchvision = {url = "https://download.pytorch.org/whl/cu111/torchvision-0.10.0%2Bcu111-cp38-cp38-linux_x86_64.whl"}

but got this:

Updating dependencies
Resolving dependencies... (326.0s)

  SolverProblemError

  Because torchvision (0.10.0+cu111) depends on torch (1.9.0)
   and  depends on torch (1.9.0+cu111), torchvision is forbidden.
  So, because  depends on torchvision (0.10.0+cu111), version solving failed.

and if I try using

 [[tool.poetry.source]]
 name = "torch_rep"
 url = "https://download.pytorch.org/whl/cu111/"

I end up disabling the pypi and can’t download anything else…

[psinger]

Coming back half a year later it seems there is still no solution for Pytorch related packages in poetry. Will this ever get solved? Otherwise need to move to other solutions.

[suneeta-mall]

This issue is blocking “unlocking the poetry” potentials when it comes to leveraging poetry in the stack that uses PyTorch and its ecosystem.

I have tried with following: toml setting

[[tool.poetry.source]]
name = "torch"
url = "https://download.pytorch.org/whl/cu113"
secondary = true
default = false

but the default false is not recognized for some reason and ends up getting the 403: Error:

403 Client Error: Forbidden for url: https://download.pytorch.org/whl/cpu/mypy

As mentioned in this ticket, https://github.com/python-poetry/poetry/issues/4704 this is a known issue. However, amongst all possible ways to address this issue, this solution of using secondary sources seems to be the ideal fix for the issue in question.

As a short-term interim, I have also tried platform and version-specific settings. This would work fine if PyTorch was my leaf dependency. Because my setup involves using PyTorch, Torchvision, and Pytorch lightening. Because more dependencies rely on PyTorch, just specifying torch wheels in toml fails to solve the dependency: toml setting

torch = [
    { url="https://download.pytorch.org/whl/cu113/torch-1.11.0%2Bcu113-cp37-cp37m-linux_x86_64.whl", python=">=3.7,<3.8", markers="sys_platform == 'linux'"},
    { url="https://download.pytorch.org/whl/cu113/torch-1.11.0%2Bcu113-cp38-cp38-linux_x86_64.whl", python=">=3.8,<3.9", markers="sys_platform == 'linux'"},
    { url="https://download.pytorch.org/whl/cu113/torch-1.11.0%2Bcu113-cp39-cp39-linux_x86_64.whl", python=">=3.9,<3.10", markers="sys_platform == 'linux'"},
    { version = "=1.11.0", markers = "sys_platform == 'darwin' or sys_platform == 'win32'" },
]
torchvision = [
    { url="https://download.pytorch.org/whl/cu113/torchvision-0.12.0%2Bcu113-cp37-cp37m-linux_x86_64.whl", python=">=3.7,<3.8", markers="sys_platform == 'linux'"},
    { url="https://download.pytorch.org/whl/cu113/torchvision-0.12.0%2Bcu113-cp38-cp38-linux_x86_64.whl", python=">=3.8,<3.9", markers="sys_platform == 'linux'"},
    { url="https://download.pytorch.org/whl/cu113/torchvision-0.12.0%2Bcu113-cp39-cp39-linux_x86_64.whl", python=">=3.9,<3.10", markers="sys_platform == 'linux'"},
    { version = "=0.12.0", markers = "sys_platform == 'darwin' or sys_platform == 'win32'" },
]

Error:

  SolverProblemError

  Because torchvision (0.12.0+cu113) depends on torch (1.11.0)
   and XXXX-app depends on torch (1.11.0+cu113), torchvision is forbidden.
  So, because XXXX-app depends on torchvision (0.12.0+cu113), version solving failed.

I have been in knots with this one, particularly because there are so many issues open around this issue: https://github.com/python-poetry/poetry/issues/2543 https://github.com/python-poetry/poetry/issues/4231 https://github.com/python-poetry/poetry/issues/3855 https://github.com/python-poetry/poetry/issues/2613 https://github.com/python-poetry/poetry/issues/4704 https://github.com/python-poetry/poetry/issues/2339

The only solution that works cross-platform is https://github.com/nat-n/poethepoet but that is not a great solution either (not lining up with lock file, not using same cache etc, the need for additional pip run!). It would be great if we can fix this issue.

(this is a duplicate comment from https://github.com/python-poetry/poetry/issues/4704#issuecomment-1109465915, posting again as this is very relevant ticket again)