Typing and public API

pytest’s “official” public API is exported by the pytest package, and everything else is defined in the _pytest package.

Currently, pytest only contains APIs that users need to use directly: functions to be called, decorators, classes to be instantiated or subclasssed, globals. Other objects, which the user doesn’t interact with directly, are not exported.

Typing extends the range of cases in which some object needs to be referred to by name. As long as the user can somehow reach an object, directly or indirectly, through some public API, they need to be able to refer to its type by name, due to type annotations.

To come up with a list of reachable but unexported types, I used the following categories:

1. Fixtures: built-in fixtures that are dependency-injected. To benefit from the types the user needs to type annotate them, for example:

   def test_it(monkeypatch: MonkeyPatch) -> None:
       ...
   

2. Hooks: the user needs to be able to type annotate the arguments and return values of any hook.

3. Transitive: types that are referred to transitively by some other public API, e.g. return value of a function, method, context manager, public attribute etc.

See below for the list I came up with.

Questions for discussion:

1. Is this a problem we want to fix?

2. Most of these types are classes, that are not intended to be instantiated and/or subclassed by users, only internally.

   2.1. Do we want to enforce this? 2.2. How do we enforce this?

3. How do we want to officialy declare something as public API?

4. Which types from the list below are we actually ready to export?

5. Is there a category or specific types I missed?

I’ll add my own thoughts on these questions in a separate reply.

---

Fixtures:

• _pytest.fixtures.FixtureRequest -> pytest.FixtureRequest
• _pytest.fixtures.SubRequest
• _pytest.cacheprovider.Cache -> pytest.Cache
• _pytest.capture.CaptureFixture -> pytest.CaptureFixture
• _pytest.logging.LogCaptureFixture -> pytest.LogCaptureFixture
• _pytest.pytester.Pytester -> pytest.Pytester
• _pytest.pytester.Testdir -> pytest.Testdir
• _pytest.tmpdir.TempdirFactory -> pytest.TempdirFactory
• _pytest.tmpdir.TempPathFactory -> pytest.TempPathFactory
• _pytest.monkeypatch.MonkeyPatch -> pytest.MonkeyPatch
• _pytest.recwarn.WarningsRecorder -> pytest.WarningsRecorder

Hooks:

• _pytest.config.Config -> pytest.Config
• _pytest.config.PytestPluginManager -> pytest.PytestPluginManager
• _pytest.config.argparsing.Parser -> pytest.Parser
• _pytest.reports.CollectReport -> pytest.CollectReport
• _pytest.python.PyCollector (Made private #9264)
• _pytest.python.Metafunc -> pytest.Metafunc
• _pytest.runner.CallInfo -> pytest.CallInfo
• _pytest.reports.TestReport -> pytest.TestReport
• _pytest.fixtures.SubRequest
• _pytest.fixtures.FixtureDef
• _pytest.terminal.TerminalReporter
• _pytest._code.code.ExceptionRepr
• _pytest.code.ExceptionInfo -> pytest.ExceptionInfo

API:

• _pytest.mark.structures.ParameterSet

Transitive:

• _pytest.fixtures.FuncFixtureInfo (thru Metafunc)
• _pytest.fixtures.FixtureFunctionMarker
• _pytest.mark.structures.MarkGenerator -> pytest.MarkGenerator
• _pytest.mark.structures.MarkDecorator -> pytest.MarkDecorator
• _pytest.mark.structures.Mark -> pytest.Mark
• _pytest.mark.structures.NodeKeywords (thru Node) - only exposed as a MutableMapping instead.
• _pytest.code._code.TerminalRepr
• _pytest.python.CallSpec2
• _pytest.python_api.ApproxBase
• _pytest.python_api.RaisesContext - Not worth exposing, just a context manager.
• _pytest.recwarn.WarningsRecorder -> pytest.WarningsRecorder
• _pytest._io.TerminalWriter
• _pytest.config.argparsing.OptionGroup -> pytest.OptionGroup
• _pytest.pytester.ParsedCall -> pytest.RecordedHookCall
• _pytest.pytester.HookRecorder -> pytest.HookRecorder
• _pytest.pytester.RunResult -> pytest.RunResult
• _pytest.pytester.LineMatcher -> pytest.LineMatcher

[gaborbernat]

An idea would be to implement declare ABCs, and only export the protocol definition. https://www.python.org/dev/peps/pep-0544/

What about using protocols instead of ABCs? And we only expose the protocols publicly 🤔 That could even be separated into a typing module:

from pytest.typing import MonkeyPatch, TempPathFactory

This way the fixtures have a very explicit public API, however the implementation does not have to inherit from it. Having the typing namespace makes it explicit that it’s not meant for instiation. E.g.:

# pytest/typing.py

class Notset:
    def __repr__(self) -> str:
        return "<notset>"


notset = Notset()

K = TypeVar("K")
V = TypeVar("V")


class MonkeyPatch(Protocol):
    @contextmanager
    def context(self) -> Generator["MonkeyPatch", None, None]:
        ...

    @overload
    def setattr(self, target: str, name: object, value: Notset = ..., raising: bool = ...) -> None:
        ...

    @overload
    def setattr(self, target: object, name: str, value: object, raising: bool = ...) -> None:
        ...

    def setattr(
        self, target: Union[str, object], name: Union[object, str], value: object = ..., raising: bool = ...,
    ) -> None:
        ...

    def delattr(self, target: Union[object, str], name: Union[str, Notset] = notset, raising: bool = ...,) -> None:
        ...

    def setitem(self, dic: MutableMapping[K, V], name: K, value: V) -> None:
        ...

    def delitem(self, dic: MutableMapping[K, V], name: K, raising: bool = ...) -> None:
        ...

[adamtheturtle]

The fixtures in the list look good to export, except maybe _pytest.fixtures.FixtureRequest / _pytest.fixtures.SubRequest which I’m not super clear about myself.

I believe _pytest.fixtures.SubRequest is what I would like to import for https://github.com/pytest-dev/pytest/issues/3342#issuecomment-665579927:

import pytest


@pytest.fixture(params=["a"])
def example_fixture(request) -> str:  # << type hint for request on this line
    return request.param