poetry startup time is slow

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: macOS cataline
• Poetry version: 1.1.4
• The contents of your pyproject.toml file:

[tool.poetry]
name = "poetry_slow_startup"
version = "0.1.0"
description = ""
authors = ["John Doe <[e-mail]>"]

[tool.poetry.dependencies]
python = "^3.9"

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

Issue

I’ll start by saying that poetry is awesome, it made me and my team more productive since we migrated from Pipenv!

I use the poetry run/shell commands a lot in my daily routine, I also use poetry as a wrapper to other tools that I use from my IDE & git hooks. However, there’s an overhead when running commands with poetry:

$ time /usr/bin/true
/usr/bin/true  0.00s user 0.00s system 69% cpu 0.003 total
$ time /usr/bin/true
/usr/bin/true  0.00s user 0.00s system 72% cpu 0.006 total
$ time /usr/bin/true
/usr/bin/true  0.00s user 0.00s system 78% cpu 0.005 total

# now with poetry

$ time poetry run /usr/bin/true
poetry run /usr/bin/true  0.54s user 0.15s system 72% cpu 0.944 total
$ time poetry run /usr/bin/true
poetry run /usr/bin/true  0.48s user 0.09s system 98% cpu 0.581 total
$ time poetry run /usr/bin/true
poetry run /usr/bin/true  0.52s user 0.14s system 78% cpu 0.844 total

It seems that running commands with poetry adds between 570-930ms latency to the command.

I tried to profile the issue, and it seems that most of the time is spent in the import statements before the main is called, the main itself is only 90ms. I tried to profile the imports with PYTHONPROFILEIMPORTTIME as follows:

PYTHONPROFILEIMPORTTIME=1 poetry run /usr/bin/true 2>imports.txt

I visualized the results using tuna as follows:

tuna imports.txt

The results indicate that most of the time is used in imports that are not used by the run/shell commands. I propose using LazyLoader to improve the startup time and responsiveness of poetry.

[lafrenierejm]

@abn, @yoav-orca May I ask what the target performance is for this issue to be considered resolved?

Based on yoav-orca’s latest comment, it looked like the timing of master at that point in time was ~350ms and similar to 1.0’s timing:

older versions of poetry were faster (poetry 1.0 is consistently under 350ms)

[jules-ch]

#3618 might help. I think we can wait & see if that improves performance.