How does `xdoctest` work with the Sphinx doctest extension?

sphinx.ext.doctest lets you write doctests in your prose documentation (e.g. .rst files), which is really useful for ensuring that usage examples in those docs do in fact still work.

If it’s possible to use xdoctest instead of the standard library doctest, that would be lovely - and could do with documentation. If not, a comparison section of the readme would be nice 🙂

[Erotemic]

FYI: I’ve started playing around with some hacky stuff that executes xdoctest on sphinx docstrings.

I have the start of this in my xcookie module: https://github.com/Erotemic/xcookie/blob/a1af10d953302eb04583bf56f6a4da76ff0b8773/docs/source/conf.py#L585

What I’m doing here is I’m using xdoctest to execute any test that creates a matplotlib figure and then it munges the sphinx output to include that image in the final docs. Such an approach can be used to simply execute them all instead.

The basic idea is: use the sphinx.ext.autodoc plugin, and then register a custom callback via the autodoc-process-docstring hook. Then you can write a function that will be passed each docstring.

A hook like this should work pretty well:

def custom_sphinx_run_doctests(app, obj, name, lines):
    import xdoctest
    import sys
    import types
    if isinstance(obj, types.ModuleType):
        module = obj
    else:
        module = sys.modules[obj.__module__]
    modpath = module.__file__
    docstr = '\n'.join(lines)

    import re
    split_parts = re.split('({}\\s*\n)'.format(re.escape('.. rubric:: Example')), docstr)

    # It would be better if xdoctest just has an RST parser. 
    doctests = list(xdoctest.core.parse_docstr_examples(
        part, modpath=modpath, callname=name,
        # style='google'
    ))

    # Not sure why this is so complex. It really shouldn't be.
    try:
        import pytest  # NOQA
    except ImportError:
        pass
    try:
        from xdoctest.exceptions import Skipped
    except ImportError:  # nocover
        # Define dummy skipped exception if pytest is not available
        class Skipped(Exception):
            pass

    for doctest in doctests:
        try:
            doctest.mode = 'native'
            doctest.run(verbose=1, on_error='raise')
        except Skipped:
            print(f'Skip doctest={doctest}')
        except Exception as ex:
            print(f'ex={ex}')
            

I could absolutely see something like this being made into a proper sphinx extension fairly easily. You could even include the real output of each doctest in the generated docs (I think the standard doctest plugin can do that too).

[tony]

Good to hear this

I went from wanting a sphinx extension, to finding it was an impediment. The dream now is pytest / pure doctest can nibble on, so it can scoop up tests in markdown and reStructuredText.

I am going to be studying doctest.py more closely over the weeks