imported TestCase does not run setUpClass on second encounter

the issue

I often write a base TestCase with some generic tests, then use this as a parent class for more derived TestCases.

When importing the base TestCase into other files pytest discovers it and runs it (as does unittest). However pytest does not process setUpClass on the base TestCase the second time that it is encountered.

an example

src/
    __init__.py
    base.py
    test_foo.py
    test_bar.py

# base.py

import unittest


class BaseTestCase(unittest.TestCase):

    @classmethod
    def setUpClass(cls):
        if cls is BaseTestCase:
            raise unittest.SkipTest("skipping base class")
        else:
            cls.foo = True

    def test_called(self):
        # should only reach here in subclass
        self.assertTrue(self.foo)

# test_foo.py

from .base import BaseTestCase


class TestFoo(BaseTestCase):
    pass

# test_bar.py

from .base import BaseTestCase


class TestBar(BaseTestCase):
    pass

expected behaviour

2 skips, 2 passes - like unittest

$ python -m unittest -v
skipped 'skipping base class'
test_called (src.test_bar.TestBar) ... ok
skipped 'skipping base class'
test_called (src.test_foo.TestFoo) ... ok

----------------------------------------------------------------------
Ran 2 tests in 0.001s

OK (skipped=2)

actual behaviour

$ pytest -v src
...
src/test_bar.py::BaseTestCase::test_called SKIPPED                                                               [ 25%]
src/test_bar.py::TestBar::test_called PASSED                                                                     [ 50%]
src/test_foo.py::BaseTestCase::test_called FAILED                                                                [ 75%]
src/test_foo.py::TestFoo::test_called PASSED                                                                     [100%]

======================================================= FAILURES =======================================================
_______________________________________________ BaseTestCase.test_called _______________________________________________

self = <src.base.BaseTestCase testMethod=test_called>

    def test_called(self):
        # should only reach here in subclass
>       self.assertTrue(self.foo)
E       AttributeError: 'BaseTestCase' object has no attribute 'foo'

src/base.py:15: AttributeError
=============================================== short test summary info ================================================
FAILED src/test_foo.py::BaseTestCase::test_called - AttributeError: 'BaseTestCase' object has no attribute 'foo'
======================================== 1 failed, 2 passed, 1 skipped in 0.07s ========================================

The fail is because the second time that BaseTestCase is encountered setUpClass is not run (which would skip it).

short term fix

a quick fix is to use from . import base to avoid a reference to BaseTestCase

environment

$ python -V
Python 3.7.6
$ pip freeze
attrs==19.3.0
certifi==2020.4.5.1
importlib-metadata==1.6.0
more-itertools==8.2.0
packaging==20.1
pluggy==0.13.0
py==1.8.1
pyparsing==2.4.7
pytest==5.4.1
six==1.14.0
wcwidth==0.1.9
zipp==3.1.0

[Zac-HD]

If pytest must 100% replicate unittest’s semantics then it’s obvious what must be done.

We want pytest to be a drop-in replacement for the unittest runner, so we’ll need to replicate that behaviour.

Thanks to everyone who has contributed to the discussion of this problem!

[pbatko]

Ignoring the imported BaseTestCase would solve the problem, but diverge from the behaviour of unittest.

If pytest must 100% replicate unittest’s semantics then it’s obvious what must be done. If it doesn’t then the simpler solution from user perspective is one from https://github.com/pytest-dev/pytest/issues/7092. That is unless there is a valid use case of running imported test classes at import sites. But even the OP gave an example where it is prevented with an exception:

raise unittest.SkipTest("skipping base class")