Adding a source even with secondary true take precedence

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Linux 5.14.12 & 5.16.0

• Poetry version: 1.1.11 & 1.2.0a2

Issue

Adding a source like

[[tool.poetry.source]]                                                          
    name = "pytorch"                                                                
    url = "https://download.pytorch.org/whl/cu111"                                    
    secondary = true   

Works perfectly well when installing torch with

torch = { version = "=1.10.0+cpu", source = "pytorch"}    

but when trying to add mypy with poetry add poetry add mypy then I got a :

403 Client Error: Forbidden for url: https://download.pytorch.org/whl/cpu/mypy

Which obviously doesn’t work as mypy is not in the pytorch package repository… I don’t get why pypi is not the default and “Pytorch” just as a fallback.

Thanks in advance for any help, Have a great day.

[mxab]

I think I’m experiencing the same issue

[suneeta-mall]

This issue is blocking “unlocking the poetry” potentials when it comes to leveraging poetry in the stack that uses PyTorch and its ecosystem.

I have tried with following: toml setting

[[tool.poetry.source]]
name = "torch"
url = "https://download.pytorch.org/whl/cu113"
secondary = true
default = false

but the default false is not recognized for some reason and ends up getting the 403: Error:

403 Client Error: Forbidden for url: https://download.pytorch.org/whl/cpu/mypy

As mentioned in this ticket, https://github.com/python-poetry/poetry/issues/4704 this is a known issue. However, amongst all possible ways to address this issue, this solution of using secondary sources seems to be the ideal fix for the issue in question.

As a short-term interim, I have also tried platform and version-specific settings. This would work fine if PyTorch was my leaf dependency. Because my setup involves using PyTorch, Torchvision, and Pytorch lightening. Because more dependencies rely on PyTorch, just specifying torch wheels in toml fails to solve the dependency: toml setting

torch = [
    { url="https://download.pytorch.org/whl/cu113/torch-1.11.0%2Bcu113-cp37-cp37m-linux_x86_64.whl", python=">=3.7,<3.8", markers="sys_platform == 'linux'"},
    { url="https://download.pytorch.org/whl/cu113/torch-1.11.0%2Bcu113-cp38-cp38-linux_x86_64.whl", python=">=3.8,<3.9", markers="sys_platform == 'linux'"},
    { url="https://download.pytorch.org/whl/cu113/torch-1.11.0%2Bcu113-cp39-cp39-linux_x86_64.whl", python=">=3.9,<3.10", markers="sys_platform == 'linux'"},
    { version = "=1.11.0", markers = "sys_platform == 'darwin' or sys_platform == 'win32'" },
]
torchvision = [
    { url="https://download.pytorch.org/whl/cu113/torchvision-0.12.0%2Bcu113-cp37-cp37m-linux_x86_64.whl", python=">=3.7,<3.8", markers="sys_platform == 'linux'"},
    { url="https://download.pytorch.org/whl/cu113/torchvision-0.12.0%2Bcu113-cp38-cp38-linux_x86_64.whl", python=">=3.8,<3.9", markers="sys_platform == 'linux'"},
    { url="https://download.pytorch.org/whl/cu113/torchvision-0.12.0%2Bcu113-cp39-cp39-linux_x86_64.whl", python=">=3.9,<3.10", markers="sys_platform == 'linux'"},
    { version = "=0.12.0", markers = "sys_platform == 'darwin' or sys_platform == 'win32'" },
]

Error:

  SolverProblemError

  Because torchvision (0.12.0+cu113) depends on torch (1.11.0)
   and XXXX-app depends on torch (1.11.0+cu113), torchvision is forbidden.
  So, because XXXX-app depends on torchvision (0.12.0+cu113), version solving failed.

I have been in knots with this one, particularly because there are so many issues open around this issue: https://github.com/python-poetry/poetry/issues/2543 https://github.com/python-poetry/poetry/issues/4231 https://github.com/python-poetry/poetry/issues/3855 https://github.com/python-poetry/poetry/issues/2613 https://github.com/python-poetry/poetry/issues/4704 https://github.com/python-poetry/poetry/issues/2339

The only solution that works cross-platform is https://github.com/nat-n/poethepoet but that is not a great solution either (not lining up with lock file, not using same cache etc, the need for additional pip run!). It would be great if we can fix this issue.