Add a built-in way to compare sequences ignoring order

What’s the problem this feature will solve?

A convenient way to assert that two sequences contains the exact same elements regardless of their order.

Describe the solution you’d like

assert [1, 2, 3] == pytest.unordered([2, 1, 3])

For example, I could use it when order of elements are not part of the specification of a function:

# Per "Path.iterdir()" documentation: "The children are yielded in arbitrary order"
assert list(tmp_path.iterdir()) == pytest.unordered([tmp_path / "some_file.txt", tmp_path / "another_file.txt"])

Alternative Solutions

Sorting the sequence manually, using unittest.TestCase.assertCountEqual, using pytest-unordered plugin.

Additional context

This issue duplicates:

• https://github.com/pytest-dev/pytest/issues/5548
• https://github.com/pytest-dev/pytest/issues/7899

These tickets were closed with the suggestion to first create an external plugin and testing it during a few years. Following this, @utapyngo created the pytest-unordered plugin. It has been several years. I would like to bring back the suggestion of a built-in pytest.unordered() helper similar to pytest.approx() because I think pytest users would greatly benefit from it.

I don’t think it opens the room for too many assertion helpers: with approx() and unordered() all main cases are covered. For comparison, here is the list of helpers provided by the Catch2 C++ testing library. Python provides many built-in functions to ease assertion, but unordered() is missing and is non-trivial.

What do you think?

[RonnyPfannschmidt]

When order doesn’t matter, comparing the result of sorted is orders of magnitude simpler to comprehend.

If unique values should be folded, comparing sets is order of magnitude simpler to comprehend.

So far i haven’t seen a setup where unordered is necessary and more comprehensive.

I’d like to see one before adding such a non trivial tool.

[kalekundert]

If it helps to have an example of unordered in a real test, here’s one I wrote just the other day: stepwise/tests/test_reaction.py. This is exactly one of the cases that @Delgan described. I’m trying to compare two lists-of-dicts that each represent the edges of a graph data structure. Because dicts are neither sortable nor hashable, neither sorted nor set can work here.

I use unordered in almost every package I write tests for, and I think it’s very useful. I’d love to see it added to pytest proper.