Breaking changes to poetry install?

Issue Summary

With the release of poetry-core 1.1.0b1, some users installing the latest Poetry pre-release (1.2.0b1) in CI environments via the install script can encounter certain issues due to either bug fixes and/or improvements in poetry-core that require related changes that are only available in Poetry master branch.

Resolution

Invalid cosntraint parsing regression has been resolved with https://github.com/python-poetry/poetry-core/releases/tag/1.1.0b2

Mitigation

Users impacted by this issue can either downgrade version of poetry-core https://github.com/python-poetry/poetry/issues/5681#issuecomment-1136143658 or use Poetry from git https://github.com/python-poetry/poetry/issues/5681#issuecomment-1136171151.

Issue (original)

Here’s how we install poetry:

wget https://install.python-poetry.org/install-poetry.py
python ./install-poetry.py --version 1.2.0b1

Our poetry.lock file contains:

[[package]]
name = "webcolors"
version = "1.11.1"
description = "A library for working with color names and color values formats defined by HTML and CSS."
category = "main"
optional = false
python-versions = ">=3.5,"

Note that the python-versions constraint seems to be malformed (extra trailing comma).

Prior to 2022-05-23 poetry install (on 1.2.0b1, with this malformed version specification) worked fine. This morning we re-installed poetry 1.2.0b1 using the same install procedure above, and we now get an error:

Could not parse version constraint: >=3.5,

Something seems to have changed out from under us even with specifying a version for poetry that is causing an error and our production pipeline to fail.

Additionally, running poetry lock --no-update on today’s re-install of 1.2.0b1 seems to be changing the category of our packages (within poetry.lock) from “main” to “dev”.

What has changed with the poetry installation, and how do we get it back? This is currently breaking production systems for us.

[abn]

In general, is there a reason that a specific release of poetry is not tied to a specific release of poetry-core?

Typically we restrict to a minor version of poetry-core as we do not normally expect things to break between pre-releases. In this case from 1.1.0a7 to 1.1.0b1 of poetry-core. This is usually sufficient restriction. As far as I can tell most of the issues reported were either misfeatures or outright bugs in 1.2.0b1 which were fixed after the release either via changes in Poetry or poetry-core. Note that this is a new minor for core and poetry simultaneously.

This caught us off guard as well. There seems to be a gap in our test matrix that does not ensure that an upcoming release of poetry-core does not break a previously released version of Poetry. We only test the current master for Poetry.

All that said, I suspect in the future, we might restrict core version a bit further, atleast prior to a stable release.

but it’s hard to justify using poetry for a production system if we could wake up any morning and find CI is suddenly failing even though no code or dependencies changed on our end.

The frustration is understandable. I suspect part of the issue here is that the 1.2.0 release has been piling on fixes and features forcing the community to use a prerelease in production environments. I am hopeful that once the 1.2.0 release is done we should be able to get new fixes out with better cadence. This should aleaviate the risk of relying on unstable and prone to breaking releases.

For environments that need a higher degree of confidence, I would even recommend that a constraints file be used when installing poetry. Hopefully though, this is merely a transitional issue.

[gavrieladler]

The issue has to do with Poetry’s runtime environment. When you install using the script, a venv is created and poetry is installed via pip. Since poetry 1.2.0b1 says Requires-Dist: poetry-core>=1.1.0a7 the latest beta gets installed.

A quick way to downgrade core is to execute this. Note this is only tested for linux environments.

${POETRY_HOME:-${XDG_DATA_HOME:-~/.local/share}}/pypoetry/venv/bin/python -m pip install --force --no-deps poetry-core==1.1.0a7

Ref: https://github.com/python-poetry/install.python-poetry.org/blob/0fab38103455fdb6d630f25ce0d4ccb11f90f9cb/install-poetry.py#L6-L14

Hi @abn, thanks for the reply and the explanation.

In general, is there a reason that a specific release of poetry is not tied to a specific release of poetry-core? I just made an internal push for my organization to move our build to poetry, but it’s hard to justify using poetry for a production system if we could wake up any morning and find CI is suddenly failing even though no code or dependencies changed on our end.