Install failure in more than two nested subprojects

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: 10.15.2 macOS
• Poetry version: 1.0.0

Issue

a
├── a
│   └── __init__.py
├── b
│   ├── b
│   │   └── __init__.py
│   ├── c
│   │   ├── c
│   │   │   └── __init__.py
│   │   └── pyproject.toml
│   └── pyproject.toml
└── pyproject.toml

5 directories, 6 files

Execute cat a/pyproject.toml

[tool.poetry]
name = "a"
version = "0.1.0"
description = ""
authors = ["Your Name <[e-mail]>"]

[tool.poetry.dependencies]
python = "^3.8"
b = {path = "b"}

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

Execute cat a/b/pyproject.toml

[tool.poetry]
name = "b"
version = "0.1.0"
description = ""
authors = ["Your Name <[e-mail]>"]

[tool.poetry.dependencies]
python = "^3.8"
c = {path="c"}

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

Execute cat a/b/c/pyproject.toml

[tool.poetry]
name = "c"
version = "0.1.0"
description = ""
authors = ["Your Name <[e-mail]>"]

[tool.poetry.dependencies]
python = "^3.8"

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

In project structure like this, execute poetry install -vvv will fail, and it’s output below

Execute poetry install -vvv

Using virtualenv: /Users/linw1995/Library/Caches/pypoetry/virtualenvs/a-lL1Z62BU-py3.8
Updating dependencies
Resolving dependencies...
   1: fact: a is 0.1.0
   1: derived: a
   1: fact: a depends on b (*)
   1: selecting a (0.1.0)
   1: derived: b (*)
   1: fact: b (0.1.0) depends on c (*)
   1: selecting b (0.1.0 b)
   1: derived: c (*)
   1: selecting c (0.1.0 b/c)
   1: Version solving took 0.057 seconds.
   1: Tried 1 solutions.

Writing lock file


Package operations: 2 installs, 0 updates, 0 removals

  - Installing c (0.1.0 b/c)

[EnvCommandError]
Command ['/Users/linw1995/Library/Caches/pypoetry/virtualenvs/a-lL1Z62BU-py3.8/bin/pip', 'install', '--no-deps', '-U', '-e', '/Users/linw1995/a/b/b/c'] errored with the following return code 1, and output:
ERROR: /Users/linw1995/a/b/b/c is not a valid editable requirement. It should either be a path to a local project or a VCS URL (beginning with svn+, git+, hg+, or bzr+).
WARNING: You are using pip version 19.2.3, however version 19.3.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.


Traceback (most recent call last):
  File "/Users/linw1995/.pyenv/versions/3.8.0/Python.framework/Versions/3.8/lib/python3.8/site-packages/clikit/console_application.py", line 131, in run
    status_code = command.handle(parsed_args, io)
  File "/Users/linw1995/.pyenv/versions/3.8.0/Python.framework/Versions/3.8/lib/python3.8/site-packages/clikit/api/command/command.py", line 120, in handle
    status_code = self._do_handle(args, io)
  File "/Users/linw1995/.pyenv/versions/3.8.0/Python.framework/Versions/3.8/lib/python3.8/site-packages/clikit/api/command/command.py", line 171, in _do_handle
    return getattr(handler, handler_method)(args, io, self)
  File "/Users/linw1995/.pyenv/versions/3.8.0/Python.framework/Versions/3.8/lib/python3.8/site-packages/cleo/commands/command.py", line 92, in wrap_handle
    return self.handle()
  File "/Users/linw1995/.pyenv/versions/3.8.0/Python.framework/Versions/3.8/lib/python3.8/site-packages/poetry/console/commands/install.py", line 63, in handle
    return_code = installer.run()
  File "/Users/linw1995/.pyenv/versions/3.8.0/Python.framework/Versions/3.8/lib/python3.8/site-packages/poetry/installation/installer.py", line 74, in run
    self._do_install(local_repo)
  File "/Users/linw1995/.pyenv/versions/3.8.0/Python.framework/Versions/3.8/lib/python3.8/site-packages/poetry/installation/installer.py", line 286, in _do_install
    self._execute(op)
  File "/Users/linw1995/.pyenv/versions/3.8.0/Python.framework/Versions/3.8/lib/python3.8/site-packages/poetry/installation/installer.py", line 302, in _execute
    getattr(self, "_execute_{}".format(method))(operation)
  File "/Users/linw1995/.pyenv/versions/3.8.0/Python.framework/Versions/3.8/lib/python3.8/site-packages/poetry/installation/installer.py", line 327, in _execute_install
    self._installer.install(operation.package)
  File "/Users/linw1995/.pyenv/versions/3.8.0/Python.framework/Versions/3.8/lib/python3.8/site-packages/poetry/installation/pip_installer.py", line 32, in install
    self.install_directory(package)
  File "/Users/linw1995/.pyenv/versions/3.8.0/Python.framework/Versions/3.8/lib/python3.8/site-packages/poetry/installation/pip_installer.py", line 221, in install_directory
    return self.run(*args)
  File "/Users/linw1995/.pyenv/versions/3.8.0/Python.framework/Versions/3.8/lib/python3.8/site-packages/poetry/installation/pip_installer.py", line 118, in run
    return self._env.run_pip(*args, **kwargs)
  File "/Users/linw1995/.pyenv/versions/3.8.0/Python.framework/Versions/3.8/lib/python3.8/site-packages/poetry/utils/env.py", line 824, in run_pip
    return self._run(cmd, **kwargs)
  File "/Users/linw1995/.pyenv/versions/3.8.0/Python.framework/Versions/3.8/lib/python3.8/site-packages/poetry/utils/env.py", line 1052, in _run
    return super(VirtualEnv, self)._run(cmd, **kwargs)
  File "/Users/linw1995/.pyenv/versions/3.8.0/Python.framework/Versions/3.8/lib/python3.8/site-packages/poetry/utils/env.py", line 856, in _run
    raise EnvCommandError(e, input=input_)

By installing in more than two nested subprojects, the second subprojects’ req value went wrong.

https://github.com/python-poetry/poetry/blob/945ca1d365d66489858e05b4b50d06e579af56ba/poetry/installation/pip_installer.py#L181

package.root_dir='a/b' is the directory of package b, package.source_url='b/c' is the relative path from package a to package c. This line of code gets a/b/b/c by joining this two values.

[kevincon]

could you please explain why you need this structure? IMO different projects should be kept separated.

I am wishing a nested project structure could be supported to help centralize tool configuration in a monorepo containing multiple Python packages while still supporting being able to publish each package individually to a pypi server.

E.g. I am working on a project with the following structure, where project-package, support-package-1, and support-package-2 are all poetry-managed Python packages:

project-package
│   mypy.ini
│   pyproject.toml
│
├───lib
│   ├───support-package-1
│   │   │   pyproject.toml
│   │   │
│   │   ├───src
│   │   │   └───support_package_1
│   │   │           ...
│   │   │
│   │   ├───stubs
│   │   │       ...
│   │   │
│   │   └───tests
│   │           ...
│   │
│   └───support-package-2
│       │   pyproject.toml
│       │
│       ├───src
│       │   └───support_package_2
│       │           ...
│       │
│       └───tests
│               ...
├───src
│   └───project_package
│           ...
│
└───tests
        ...

The dependency hierarchy I would like to have in this setup is that root-package depends on support-package-1, and support-package-1 depends on support-package-2. However, this doesn’t seem to be possible today with poetry 1.0.9 on Windows 10. I get the following error when trying to run poetry update from the root of root-package with that dependency hierarchy encoded in the pyproject.toml files:

Package operations: 10 installs, 0 updates, 0 removals

  - Installing support-package-2 (0.1.0 lib/support-package-2)

[EnvCommandError]
Command ['C:\\Users\\kdconley\\AppData\\Local\\pypoetry\\Cache\\virtualenvs\\root-package-NG3dGAKP-py3.8\\Scripts\\pip.exe', 'install', '--no-deps', '-U', '-e', 'C:\\Users\\kdconley\\root-package\\lib\\support-package-1\\lib/support-package-2'] errored with the following return code 1, and output:
ERROR: C:\Users\kdconley\root-package\lib\support-package-1\lib/support-package-2 is not a valid editable requirement. It should either be a path to a local project or a VCS URL (beginning with svn+, git+, hg+, or bzr+).

For now I am working around this limitation by explicitly adding both support-package-1 and support-package-2 to the list of dependencies in root-package’s pyproject.toml file. That seems to make poetry update happy. But this is technically unnecessary since the knowledge that support-package-1 depends on support-package-2 is already encoded in support-package-1’s pyproject.toml file.

As an example for what I mean by “centralize tool configuration”, I have written the mypy.ini file in the root of the root-package directory to enable easily type-checking all of these nested packages by running poetry run mypy from the root of root-package:

# mypy.ini
[mypy]
; Type-check all Python files under root-package including those in support-package-1 and support-package-2
files = **/*.py
; Use some stubs only needed for support-package-1, etc.
mypy_path = lib/support-package-1/stubs
..

[linw1995]

Recently I found out that Poetry version 1.1.2 fixes this issue. Thanks to the effort that all contributors made.