allure-pytest does not work with pytest-check

When using pytest-check to have delayed assertions in tests, allure marks all the tests as passed even when the delayed checks are false. After a little investigation it looks to be because of this check_func dec:

def check_func(func):
    @functools.wraps(func)
    def wrapper(*args, **kwds):
        __tracebackhide__ = True
        try:
            func(*args, **kwds)
        except AssertionError as e:
            if _stop_on_fail:
                raise e
            log_failure(e)
            return False

    return wrapper

to recreate:

pip install pytest-check

    import pytest_check as check
    @allure.title('This test will pass on allure report')
    def test_something(self):
        check.equal(1, 2)

allure serve # voila, report is green 😦

I guess because it is catching the AssertionError and logging it elsewhere. The actual pytest test is a failure, however allure reports a pass. No time to look into this right now, but any advice on where to start with it will be appreciated

[symonk]

@skhomuti / @okken sorry been busy with other commitments, happy to spin up a venv and check, 1 second.

Edit: Can confirm it seems patched

dependencies:

allure-pytest==2.9.43
allure-python-commons==2.9.43
attrs==21.2.0
iniconfig==1.1.1
packaging==21.0
pluggy==0.13.1
py==1.10.0
pyparsing==2.4.7
pytest==6.2.4
pytest-check==1.0.2
six==1.16.0
toml==0.10.2

allure commandline -> 2.14.0

import allure


@allure.story("Foo!")
def test_bar(check):
    check.equal(1,2)

Report is red as expected.

I have also removed my previous comment recommending another library as this did not originally work, as I don’t think it was very fair in hindsight (I apologise for that), thanks for your work on pytest-check. Would recommend exposing check w/o an explicit name= flag to help IDEs recommend it a little better, didn’t look overly at the code but:

@pytest.fixture
def check():
    return check_methods

would make auto complete a lot happier

[sseliverstov]

Thank you @skhomuti !