Running with ipython 8 results in crash

Issue

Usint ipytest with latest ipython 8.0.1 results in program crash with ModuleNotFoundError.

Example

Simple test notebook

ipytest_nb.ipynb

import ipytest
ipytest.autoconfig()

def test_fail():
    assert False
ipytest.run()

IPython 7.23.1

(venv) rschulze@red: (dev *)~/development/pyshl$ ipython --TerminalIPythonApp.file_to_run=ipytest_nb.ipynb
F                                                                                            [100%]
============================================= FAILURES =============================================
____________________________________________ test_fail _____________________________________________

    def test_fail():
>       assert False
E       assert False

<ipython-input-1-ff73e06d7454>:2: AssertionError
===================================== short test summary info ======================================
FAILED tmp0n4g6iqq.py::test_fail - assert False
1 failed in 0.05s

IPython 8.0.1

(venv) rschulze@red: (dev *)~/development/pyshl$ ipython --TerminalIPythonApp.file_to_run=ipytest_nb.ipynb

ModuleNotFoundError                       Traceback (most recent call last)
~/development/pyshl/venv/lib/python3.8/site-packages/runipy/notebook_runner.py in <module>
     26         # IPython 3
---> 27         from IPython.kernel import KernelManager
     28         from IPython.nbformat import NotebookNode

ModuleNotFoundError: No module named 'IPython.kernel'

[…]

Solution

It seems that following replacements all-over the code solve the issue:

1. from IPython.nbformat import… -> from nbformat import…
2. from IPython.kernel import… -> jupyter_client import…

Probably have to be embedded in some try:... except:... for backwards compatibility.

[ralfschulze]

Are you sure about the ipytest.run() == 0 line? In my tests the run() method always give None and there does not seem to be a return statement in run() https://github.com/chmp/ipytest/blob/b9d8dff02fddafe0f9a1742b76cbac20d406994a/ipytest/_impl.py#L22

What I want to achieve is to run automated unit-tests in my notebook when I push it to our CI/CD.

I tried to run the notebook via ipython --InteractiveShellApp.code_to_run='%run TestNotebook.ipynb. That works in the sense that all assertions are reported, but ipython does not set an exit code != 0 when assertions fail, so this does not work for CI/CD.

Your solution via pytest --nbval-lax --current-env TestNotebook.ipynb gives the clear pytest output, but I have to put assert ipytest.exit_code == 0 at the end of each cell, or the tests will always pass. I would prefer to have this checked automatically, but the line is easy enough to add (Just have to remember it 😃, so I will try to use this solution for my problem.

Anyhow: Thanks for sharing this package and also for your support!

[chmp]

@ralfschulze you’re right, the return code is not returned. I released ipytest==0.12.0b1 which offers two options to make sure test failures lead to exceptions (see https://github.com/chmp/ipytest/issues/63):

1. Use ipytest.autoconfig(raise_on_error=True)
2. Use assert ipytest.run() == 0

It would be cool, if you could proide feedback, whether the changes work for your use case.