Path dependencies are slow to install

Prereqs

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Windows 10 + WSL (Ubuntu 18.04.1 LTS)

• Poetry version: tried with 0.9.0 and 1.0.0a5

• Pip version: tried with 18.1 and 19.2.1

Issue

Installation of path dependencies during poetry install is very slow compared to installation of PyPi dependencies.

Sample pyproject.toml file:

[tool.poetry.dependencies]
python = "^3.6"
pyyaml = "^3.13"
xyz-package1 = {path = "../../packages/xyz-package1"}
xyz-package2 = {path = "../../packages/xyz-package2"}
xyz-package3 = {path = "../../packages/xyz-package3"}

Measuring the time to install each individual dependency with my current setup:

• PyPi dependencies: each takes between 1 and 4 seconds to install
• Local path dependencies: each takes between 21 and 26 seconds to install

Additionally, each poetry install or poetry update re-installs all path dependencies even with no changes to version or filesystem contents:

$ poetry install | ts '[%Y-%m-%d %H:%M:%S]'
[2019-07-27 15:34:58] Installing dependencies from lock file
[2019-07-27 15:34:58]
[2019-07-27 15:34:58]
[2019-07-27 15:34:58] Package operations: 0 installs, 16 updates, 0 removals
[2019-07-27 15:34:58]
[2019-07-27 15:34:58]   - Updating xyz-package1 (0.0.0 a3ed79a -> 0.1.0 ../../packages/xyz-package1)
[2019-07-27 15:35:23]   - Updating xyz-package2 (0.0.0 a3ed79a -> 0.1.0 ../../packages/xyz-package2)
[2019-07-27 15:35:46]   - Updating xyz-package3 (0.0.0 a3ed79a -> 0.1.0 ../../packages/xyz-package3)
...

My project currently has 40 PyPi dependencies and 16 path dependencies, which means that each poetry install takes around 6-7 minutes to complete.

With a bit of digging, I believe the code is generating a setup.py file and then executing pip install --no-deps -U -e <path>. Manually running this command with the addition of -v shows pip is running a lot of poetry-related operations, which don’t appear in setup.py at all - so I suspect that pip is still picking up the pyproject.toml file (which isn’t necessary, right?)

Looking into masonry/builders/editable.py now, the code in here actually renames the pyproject.toml file to pyproject.toml.tmp before executing pip, I suspect for this very reason. I can also see a number of methods for generating egg-info and egg-link files manually within editable.py. Is the plan to enable these methods or other mechanisms (eg. symlinking) in the future and avoid the use of pip install -e?

As an interim solution, is there currently any way to:

1. Speed up installation of path dependencies?
2. Prevent re-install of all path dependencies during poetry install or poetry update?

Thanks and regards, Arrian

[janjur]

It’s very slow on WSL. Project with ~30 deps ha been resolving deps for >10 minutes and it keeps going. E: It took 15 minutes.

[finswimmer]

I’m closing this because there were no more comments for a long time. Poetry evolves a lot since this issue was open. Feel free to open a new one if you still encounter problems.