inconsistent behavior report vs collect in Pytest 7 (corner case)

test example should include two test functions having the same name (this is just an example):

def test_1():
    assert False

def test_2():
    assert True

def test_1():
    assert True

Pytest 7:

platform linux -- Python 3.10.2, pytest-7.0.1, pluggy-1.0.0 -- /somedir/python
cachedir: .pytest_cache
rootdir: somedir, configfile: setup.cfg
collected 2 items                                                                    

test.py::test_1 PASSED                                                         [ 50%]
test.py::test_2 PASSED                                                         [100%]

Pytest 6:

platform linux -- Python 3.10.2, pytest-6.2.5, py-1.11.0, pluggy-1.0.0 -- /somedir/python
cachedir: .pytest_cache
rootdir: somedir, configfile: setup.cfg
collected 2 items                                                                    

test.py::test_2 PASSED                                                         [ 50%]
test.py::test_1 PASSED                                                         [100%]

With Pytest 7 tests are ordered by definition ignoring possible re-definition. I can’t find documented behavior for duplicated test functions, so feel free to close this ticket if this is something not supported.

[asottile]

in pytest 6 there was an error prone attempt to find the location of the test by loading the tests’ source and traversing through the ast and source

in pytest 7 the ordering is what cpython tells us it is (due to dictionaries maintaining insertion order and the module ~essentially being an object with __dict__ having its members)

the change in behaviour “makes sense” here – but the code seems wrong (duplicate definitions are going to clobber themselves) – so to me this seems “garbage in garbage out”

[asottile]

flake8 (via pyflakes) catches this for instance