Run tests using `unittest` and `test.support` with pytest

a detailed description of the bug or problem you are having

pytest runs fine for most unittest tests, but does not seem to work well for those that use Python’s internal modules, test and test.support.

I ran into such a test in berkeleydb’s test. (See below.)

$ pytest
==================================================== test session starts =====================================================
platform darwin -- Python 3.9.10, pytest-7.0.1, pluggy-1.0.0
rootdir: /Users/eggplants/prog/berkeleydb-18.1.5
plugins: subtests-0.7.0, cov-3.0.0
collected 0 items / 26 errors

=========================================================== ERRORS ===========================================================
_____________________________________________ ERROR collecting tests/test_all.py _____________________________________________
ImportError while importing test module '/Users/eggplants/prog/berkeleydb-18.1.5/tests/test_all.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../.pyenv/versions/3.9.10/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_all.py:51: in <module>
    from test.support.socket_helper import find_unused_port
E   ModuleNotFoundError: No module named 'test.support'; 'test' is not a package
...

I would like to be able to run pytest on such tests if possible. It enables to execute pytest-monkeytype for collecting actual types from tests.

output of pip list from the virtual environment you are using

$ pip list
Package                       Version
----------------------------- ---------
alabaster                     0.7.12
attrs                         21.4.0
Babel                         2.9.1
berkeleydb                    18.1.5
black                         21.9b0
certifi                       2021.10.8
charset-normalizer            2.0.12
click                         8.0.4
coverage                      6.3.2
doctest-ignore-unicode        0.1.2
docutils                      0.17.1
flake8                        4.0.1
flake8-black                  0.2.4
html5lib                      1.1
idna                          3.3
imagesize                     1.3.0
importlib-metadata            4.11.2
iniconfig                     1.1.1
isodate                       0.6.1
isort                         5.10.1
Jinja2                        3.0.3
markdown-it-py                2.0.1
MarkupSafe                    2.1.0
mccabe                        0.6.1
mdit-py-plugins               0.3.0
mdurl                         0.1.0
mypy                          0.940
mypy-extensions               0.4.3
myst-parser                   0.17.0
networkx                      2.7.1
nose                          1.3.7
packaging                     21.3
pathspec                      0.9.0
pbr                           5.8.1
pip                           22.0.4
platformdirs                  2.5.1
pluggy                        1.0.0
py                            1.11.0
pycodestyle                   2.8.0
pyflakes                      2.4.0
Pygments                      2.11.2
pyparsing                     3.0.7
pytest                        7.0.1
pytest-cov                    3.0.0
pytest-subtests               0.7.0
pytz                          2021.3
PyYAML                        6.0
regex                         2022.3.2
requests                      2.27.1
setuptools                    60.9.3
six                           1.16.0
snowballstemmer               2.2.0
Sphinx                        4.4.0
sphinxcontrib-apidoc          0.3.0
sphinxcontrib-applehelp       1.0.2
sphinxcontrib-devhelp         1.0.2
sphinxcontrib-htmlhelp        2.0.0
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-kroki           1.3.0
sphinxcontrib-qthelp          1.0.3
sphinxcontrib-serializinghtml 1.1.5
toml                          0.10.2
tomli                         1.2.3
types-setuptools              57.4.10
typing_extensions             4.1.1
urllib3                       1.26.8
webencodings                  0.5.1
zipp                          3.7.0

pytest and operating system versions

$ pytest --version
pytest 7.0.1
$ uname -vorm
21.3.0 Darwin Kernel Version 21.3.0: Wed Jan  5 21:37:58 PST 2022; root:xnu-8019.80.24~20/RELEASE_X86_64 x86_64 Darwin

minimal example if possible

https://pypi.org/project/berkeleydb/#files

from berkeleydb-18.1.5/tests/test_all.py:

"""
Copyright (c) 2008-2022, Jesus Cea Avion <[e-mail]>
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:

    1. Redistributions of source code must retain the above copyright
    notice, this list of conditions and the following disclaimer.

    2. Redistributions in binary form must reproduce the above
    copyright notice, this list of conditions and the following
    disclaimer in the documentation and/or other materials provided
    with the distribution.

    3. Neither the name of Jesus Cea Avion nor the names of its
    contributors may be used to endorse or promote products derived
    from this software without specific prior written permission.

    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
    CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
    INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
    MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
    DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS
    BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
    EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
            TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
            DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
    ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR
    TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
    THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
    SUCH DAMAGE.
    """

"""Run all test cases.
"""

import sys
import os, os.path
import unittest
import threading
import platform
import pathlib

import berkeleydb
from berkeleydb import db, dbutils, dbshelve, \
        hashopen, btopen, rnopen, dbobj

if sys.version_info >= (3, 9):
    from test.support.socket_helper import find_unused_port
else:
    from test.support import find_unused_port

if sys.version_info >= (3, 10):
    from test.support.os_helper import rmtree, unlink
else:
    from test.support import rmtree, unlink
...

[The-Compiler]

It doesn’t seem to work for me via unittest either:

$ python3 -m unittest test_x.py 
E
======================================================================
ERROR: test_x (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_x
Traceback (most recent call last):
  File "/usr/lib/python3.10/unittest/loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "/home/florian/test_x.py", line 1, in <module>
    from test.support.socket_helper import find_unused_port
ModuleNotFoundError: No module named 'test'


----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (errors=1)

Does it for you? After installing python-tests (which seems to be how to get the test package on Archlinux, split from python), it then works fine for unittest and pytest (and also pure Python).

I still don’t see how anything about this would be pytest-specific.

[The-Compiler]

Yay! You might want to make berkeleydb maintainers aware of the test docs, though:

Note: test.support is not a public module. It is documented here to help Python developers write tests. The API of this module is subject to change without backwards compatibility concerns between releases.

The find_unused_port function they are using is pretty much this:

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
    sock.bind(('localhost', 0))
    return sock.getsockname()[1]

and I’m pretty sure the rmtree and unlink helpers are similarly simple using pathlib or shutil…

Doesn’t look they have a bugtracker however, only a mailinglist.