Extras with dev-dependences are not recognised by `pip install`

• I am on the latest Poetry version.

• ~I have searched the issues of this repo and believe that this is not a duplicate.~ This is a duplicate to #1145.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: macOS 10.15.7

• Poetry version: 1.1.4

• Python version: 3.8.6

Issue

This is a duplicate of #1145 but, given how confusing the original thread has become, I opted to created a new issue. I’m experiencing the exact same problem as @devkral (original thread’s OP), namely: pip install .[docs] does not install extras listed under [tool.poetry.extras] when the corresponding package are listed under tool.poetry.dev-dependencies.

As @paveldedik noted in the original thread, this issue disappears when the extra dependencies are moved to tool.poetry.dependencies.

Setup

Here is my pyproject.toml (real world project, thus the verbosity):

[tool.poetry]
# ...

[tool.poetry.dependencies]
python = ">=3.6, <3.9"
attrs = "~20.3.0"
click = "~7.1.2"
click-plugins = "~1.1.1"
matplotlib = "~3.3.2"
scipy = "~1.5.4"
Shapely = {extras = ["vectorized"], version = "~1.7.1"}
svgwrite = "~1.4"
toml = "~0.10.2"
svgpathtools = {git = "https://github.com/abey79/svgpathtools", rev = "vpype"}

[tool.poetry.dev-dependencies]
pytest = "^6.1.2"
black = "^20.8b1"
isort = "^5.6.4"
# the following must be moved to [tool.poetry.dependencies] to be recognised by pip
Sphinx = { version = "^3.3.0", optional = true }
sphinx-click = {git = "https://github.com/abey79/sphinx-click", rev = "vpype-docs", optional = true}
sphinx-autodoc-typehints = { version = "^1.11.1", optional = true }
sphinx-rtd-theme = { version = "^0.5.0", optional = true }

[tool.poetry.extras]
docs = ["Sphinx", "sphinx-click", "sphinx-autodoc-typehints", "sphinx-rtd-theme"]

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

Expected behaviour

$ pip install .[docs]
[...] output indicating that pip tries to install sphinx, sphinx-click, sphinx-autodoc-typehints, sphinx-rtd-theme

Actual behaviour

pip install .[docs] does not attempt to install docs packages.

Workaround

Move docs packages to [tool.poetry.dependencies], then pip install .[docs] will correctly attempt (and succeed in this case) to install said packages.

[finswimmer]

I totally agree with @sinoroc. The dev-dependencies are only meant for dependencies during development with poetry. So if you use pip for installing the package, only the normal dependencies will be presented to pip to install.

What you can do is, defining to dependencies in the dependencies section and dev-dependencies section, like this:

[tool.poetry.dependencies]
python = ">=3.6, <3.9"
attrs = "~20.3.0"
click = "~7.1.2"
click-plugins = "~1.1.1"
matplotlib = "~3.3.2"
scipy = "~1.5.4"
Shapely = {extras = ["vectorized"], version = "~1.7.1"}
svgwrite = "~1.4"
toml = "~0.10.2"
svgpathtools = {git = "https://github.com/abey79/svgpathtools", rev = "vpype"}
Sphinx = { version = "^3.3.0", optional = true }
sphinx-click = {git = "https://github.com/abey79/sphinx-click", rev = "vpype-docs", optional = true}
sphinx-autodoc-typehints = { version = "^1.11.1", optional = true }
sphinx-rtd-theme = { version = "^0.5.0", optional = true }

[tool.poetry.dev-dependencies]
pytest = "^6.1.2"
black = "^20.8b1"
isort = "^5.6.4"
Sphinx = { version = "^3.3.0"}
sphinx-click = {git = "https://github.com/abey79/sphinx-click", rev = "vpype-docs"}
sphinx-autodoc-typehints = { version = "^1.11.1"}
sphinx-rtd-theme = { version = "^0.5.0"}

[tool.poetry.extras]
docs = ["Sphinx", "sphinx-click", "sphinx-autodoc-typehints", "sphinx-rtd-theme"]

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

Than poetry will install the dev-dependencies on poetry install. When using pip the optional packages will only gets installed if you select the corresponding extra.

fin swimmer

[henhuy]

For me, following setup gives me pip warning WARNING: unknown 0.0.0 does not provide the extra 'docs' when trying to install it via pip install .[docs]. As I read from discussion above, this should work, right?

[tool.poetry.dependencies]
python = ">=3.7, <3.9"
pandas = "0.25.3"
numpy = "1.17.5"
rtree = "0.9.7"
pyyaml = "6.0"
snakemake = "<6.0.0"
Sphinx = {version = "^4.3.2", optional = true}
sphinx-rtd-theme = {version = "^1.0.0", optional = true}
sphinxcontrib-bibtex = {version = "^2.4.1", optional = true}

[tool.poetry.dev-dependencies]
black = "20.8b1"
pytest = "^7.0.1"
coverage = "^6.3.2"
flake8 = "^4.0.1"

[tool.poetry.extras]
docs = ["Sphinx", "sphinx-rtd-theme", "sphinxcontrib-bibtex"]