skipping: possible improvements to pytest.mark.skipif/xfail condition handling

The @pytest.mark.skipif and @pytest.mark.xfail can be made conditional by passing conditions into them (usually just one), e.g.

import pytest
import sys

@pytest.mark.xfail(condition=sys.platform == "win32", reason="shouldn't work on Windows")
def test_expected_failure():
    assert False  # ...

pytest supports two flavors of conditions:

• A string - in this case the string is eval’d with an environment consisting of:

  • The os, sys and platform Python modules
  • The pytest config
  • If the Item is a Python Item, the underlying object’s __globals__ – usually its the globals of the module containing the item.

• A simple boolean, like in the example above.

IMO, the string conditions are inferior in all respects compared to boolean conditions, except for two things:

1. The condition string can be used as the default reason string, so supplying a reason is not required.
2. It has access to the config. A boolean condition is usually evaluated during import time (by normal Python rules), so doesn’t have access to it.

In #7388 @RonnyPfannschmidt suggested a possible fix for (2): allow the condition to be a function/lambda, which takes config, e.g.

import pytest

@pytest.mark.xfail(condition=lambda config: config.getoption("foo", False), reason="")
def test_expected_failure():
    assert False  # ...

This doesn’t fix (1), but that’s not too bad anyway.

I think with this feature, string conditions become redundant and can be deprecated (though we perhaps wouldn’t want to actually deprecate as that would cause a lot of churn).

An alternative is to pass item instead of config. This makes it possible to conditionalize on item properties, and the config is still accessible through item.config.

[tuukkamustonen]

There was a duplicate in https://github.com/pytest-dev/pytest/issues/9650.

Basically the same, but supporting injection of fixtures in the callable would be really nice.

In my case, I’m loading “environment-specific” configuration on-the-fly in a fixture, and would need access to that data in the condition.

[nicoddemus]

👍 to the idea of allowing functions.

Also agree to not deprecating the string variant, it would cause a ton of breakage without really simplifying our code much.

About the signature, both config and item seem reasonable, but we might decide to add more parameters in the future.

Perhaps we can extract/borrow the code from pluggy which only passes the declared arguments to functions, for forward compatibility? Then we don’t need to worry about adding more parameters later.