Combining `pytest.skip` and `request.getfixturevalue` in multiple `pytest_runtest_setup`s causes AssertionError during TearDown

Summary:

I'm using 2 plugins, each of which have `pytest_runtest_setup` hooks. One is used to get a fixture, and skip the test based on the value of a fixture. The second uses the `record_xml_attribute` fixture to add some attributes to my junitxml file (even if the test is skipped). The result of this is an AssertionError during TearDown. However, as seen in my example code, this is reproducible with any `function` and `session` scoped fixtures.

Assert being hit: _pytest/runner.py line 328

assert (colitem is None or colitem in self.stack or                                                                                                                                                   
                      isinstance(colitem, tuple))

The assertion is failing because colitem is of type _pytest.main.Session

pip list:

Package        Version
-------------- -------
atomicwrites   1.1.5  
attrs          18.1.0 
delayed-assert 0.1    
more-itertools 4.3.0  
pip            10.0.1 
pluggy         0.7.1  
py             1.5.4  
pytest         3.8.2  
setuptools     39.0.1 
six            1.11.0 

OS: Ubuntu 16.04

Minimal Example:

conftest.py:

import pytest                                                                                                                                                                                                       
                                                                                                                                                                                                                    
def pytest_runtest_setup(item):                                                                                                                                                                                     
    item._request.getfixturevalue('function_fixture')                                                                                                                                                               
    pytest.skip('Skip Reason')                                                                                                                                                                                      
                                                                                                                                                                                                                    
                                                                                                                                                                                                                    
@pytest.fixture                                                                                                                                                                                                     
def function_fixture():                                                                                                                                                                                             
    pass

tests/conftest.py

                                                                                                                                                                                                                    
def pytest_runtest_setup(item):                                                                                                                                                                                     
    item._request.getfixturevalue('session_fixture')                                                                                                                                                                
                                                                                                                                                                                                                    
                                                                                                                                                                                                                    
@pytest.fixture(scope='session')                                                                                                                                                                                    
def session_fixture(request):                                                                                                                                                                                       
    pass

tests/test_thing.py

def test_something():                                                                                                                                                                                               
    pass

Running py.test -svv . from the base directory results in an AssertionError being raised from /_pytest/runner.py line 328, in _teardown_with_finalization

[RonnyPfannschmidt]

i believe the flow is completely unsupported and its entirely sensible we fall flat, however i haven’t seen the actual original traceback ^^

[asottile]

@mctwynne this appears to work, but I don’t know your usecase to know if it satisfies it:

import pytest


x = []

@pytest.fixture
def sometimes_fixture():
    x[:] = [1, 2, 3]
    yield
    x[:] = []


@pytest.fixture(autouse=True)
def autofix(request):
    marker_names = {m.name for m in request.node.iter_markers()}
    if 'sometimes_please' in marker_names:
        request.getfixturevalue('sometimes_fixture')
    if 'but_also_skip_ok' in marker_names:
        pytest.skip('plz skip!')


@pytest.mark.sometimes_please
def test_1():
    assert x == [1, 2, 3]


def test_2():
    assert x == []


@pytest.mark.sometimes_please
@pytest.mark.but_also_skip_ok
def test_3():
    raise AssertionError('I should have been skipped!')

 pytest test.py
============================= test session starts ==============================
platform linux -- Python 3.6.6, pytest-3.8.2, py-1.7.0, pluggy-0.7.1
rootdir: /tmp/pytest, inifile: tox.ini
collected 3 items                                                              

test.py ..s                                                              [100%]
=========================== short test summary info ============================
SKIP [1] /tmp/pytest/test.py:20: plz skip!

===================== 2 passed, 1 skipped in 0.03 seconds ======================