matrix testing

This library looks like exactly what i’m looking for!

I’ve got a slightly hacky setup trying to test a matrix of both Python versions and versions of specific packages, using Nox and Poetry.

Currently looks like this (i’m still playing with it)

import nox
import tempfile
from pathlib import Path
import os
from distutils.dir_util import copy_tree

python_versions = [
    "3.5",
    "3.6",
    "3.7",
    "3.8",
]

sphinx_versions = [
    "~2.2",
    "~2.3",
    "~2.4",
    "~3.0",
    "~3.2",
]

@nox.session(python=python_versions, reuse_venv=True)
@nox.parametrize("sphinx", sphinx_versions)
def test(session, sphinx):
    # see https://github.com/python-poetry/poetry/issues/2920#issuecomment-693147409
    with tempfile.TemporaryDirectory() as tmp_dir:
        copy_tree(
            Path.cwd(), tmp_dir
        )
        os.chdir(tmp_dir)
        
        session.run('poetry', 'add', f'sphinx@{sphinx}', external=True)
        session.run('poetry', 'install', external=True)
        session.run('nosetests', '-w', 'tests')

@nox.session(python=None)
def lint(session):
    session.run('flake8', 'sphinxcontrib', external=True)

seems to work, but i don’t love it. How would you express this idiomatically using your library?

[cjolowicz]

hi @danieleades ,

Thanks for raising this use case! We actually had this conversation before 😉 see https://github.com/theacodes/nox/issues/347

An issue in nox-poetry’s support for parametrized sessions was fixed in 0.8.1, released today. You’ll need to upgrade to get this to work.

How would you express this idiomatically using your library?

Just install your package, then install the desired Sphinx version on top:

import nox
import nox_poetry

@nox_poetry.session
@nox.parametrize("sphinx", ["2.4.4", "3.4.3"])
def test(session, sphinx):
    session.install(".")
    session.run("pip", "install", f"sphinx=={sphinx}")
    session.run(
        "python",
        "-c",
        "from importlib.metadata import version; sphinx = version('sphinx'); print(f'{sphinx = }')",
    )

(Unfortunately, you cannot simply do session.install(".", f"sphinx=={sphinx}"). It seems pip does not let you override constraints files like this.)

If you want a less verbose output from the session, you can pass silent=True to session.run, which is what session.install does by default. However, the pip output may be valuable as you are installing into an already populated virtualenv.

Would that work for your use case?

@brechtm

I have dependabot set up for my project, which automatically informs me of these updates and I can update the lockfile to test against these newer versions.

In my experience, Dependabot always updates the version constraint in pyproject.toml to require the latest version, even when the existing version constraint includes the new version. AFAIU, Dependabot has support for widening version ranges, but not for Python projects. See https://docs.github.com/en/github/administering-a-repository/configuration-options-for-dependency-updates#versioning-strategy

it would be great if nox-poetry could offer the possibility of parameterizing my sessions according to the version constraints specified in pyproject.toml.

Do you (and @danieleades) see anything additional that nox-poetry could/should do to support this use case?

[brechtm]

Just install your package, then install the desired Sphinx version on top:

That seems like it would work perfectly 👍

In my experience, Dependabot always updates the version constraint in pyproject.toml to require the latest version, even when the existing version constraint includes the new version. AFAIU, Dependabot has support for widening version ranges, but not for Python projects.

I also saw that in the GitHub docs, but it does seem to be widening the version range in practice.

Do you (and @danieleades) see anything additional that nox-poetry could/should do to support this use case?

No, I think with the information in this thread it should be possible to get this to work.

Of course, it would be great if nox-poetry could include the functionality I discussed above, but that’s of course up to you to decide! Or perhaps you would accept a pull request?

I’ve had a go at implementing the get_versions() function. I’m not sure when I’ll have time to transform this into a proper implementation, so I’ll just leave it here for future reference:

import json
from collections.abc import Iterable
from pathlib import Path
from typing import Optional
from urllib.request import urlopen, Request

from poetry.core.factory import Factory
from poetry.core.semver import parse_single_constraint as parse_version


VERSION_PARTS = ('major', 'minor', 'patch')


def get_versions(dependency: str, granularity: str = 'minor',
                 # ascending: bool = False, limit: Optional[int] = None,
                 # allow_prerelease: bool = False,
                 ) -> Iterable[str]:
    """Yield all versions of `dependency` considering version constraints

    Args:
        dependency: the name of the dependency
        granularity: yield only the newest patch version of each major/minor
            release
        ascending: count backwards from latest version, by default (not much
            use without the 'limit' arg)
        limit: maximum number of entries to return
        allow_prerelease: whether to include pre-release versions

    Yields:
        All versions of `dependency` that match the version constraints defined
        and in this project's pyproject.toml and the given `granularity`.

    """
    package = Factory().create_poetry(Path(__file__).parent).package
    for requirement in package.requires:
        if requirement.name == dependency:
            break
    else:
        raise ValueError(f"{package.name} has no dependency '{dependency}'")
    constraint = requirement.constraint
    filtered_versions = [version for version in all_versions(dependency)
                         if constraint.allows(version)]
    parts = VERSION_PARTS[:VERSION_PARTS.index(granularity) + 1]
    result = {}
    for version in filtered_versions:
        key = tuple(getattr(version, part) for part in parts)
        result[key] = (max((result[key], version))
                       if key in result else version)
    yield from (str(version) for version in result.values())


def all_versions(dependency):
    request = Request(f'https://pypi.org/pypi/{dependency}/json')
    response = urlopen(request)
    json_string = response.read().decode('utf8')
    json_data = json.loads(json_string)
    yield from (parse_version(version) for version in json_data['releases'])

You need to save this code to a file in the same directory as your pyproject.toml or adjust the path passed to create_poetry().

With a dependency sphinx >= 2.2.1, this returns:

• get_versions('sphinx', 'major'): 2.4.4, 3.4.3
• get_versions('sphinx', 'minor'): 2.2.2, 2.3.1, 2.4.4, 3.0.4, 3.1.2, 3.2.1, 3.3.1, 3.4.3
• get_versions('sphinx', 'patch'): 2.2.1, 2.2.2, 2.3.0, 2.3.1, 2.4.0, 2.4.1, 2.4.2, 2.4.3, 2.4.4, 3.0.0, 3.0.1, 3.0.2, 3.0.3, 3.0.4, 3.1.0, 3.1.1, 3.1.2, 3.2.0, 3.2.1, 3.3.0, 3.3.1, 3.4.0, 3.4.1, 3.4.2, 3.4.3