Programmatically exit a notebook early when match some condition

Is your feature request related to a problem? Please describe. A clear and concise description of what the problem is. Ex. I’m always frustrated when […]

Programmatically exit a notebook early .

Describe the solution you’d like

I have below code use to skip a notebook node:

class StopExecution(Exception):
    def _render_traceback_(self):
        pass

def exit_notebook(): raise StopExecution

if vstore.exists(Config.model_name):
    exit_notebook()

But would got error when run in pipeline : Failed validating 'type' in notebook['properties']['traceback']:

Describe alternatives you’ve considered

I don’t have a clear solution now. Since elyra doesn’t provide a way to avoid redundant execution, I tried to make it myself, but failed by avoid redundant execution. Do you have a way to walk around ?

[ptitzler]

If your ultimate goal is to skip execution of some cells without causing papermill to return an error you might be able to accomplish that using

import sys
sys.exit(0)

If this code is somewhere in the first notebook and triggered then the second notebook will still run.

I’ve tested this using local execution and Kubeflow Pipelines.

You can combine this with the following code to assure that this only happens when you run the notebook in a pipeline and not when you run it interactively:

if os.environ.get("ELYRA_RUNTIME_ENV") is not None:
    sys.exit(0)

The proprietary ELYRA_RUNTIME_ENV environment variable is only defined when Elyra executes a notebook or script.

You do need to be aware of that stopping the interpreter might have unintended consequences.

[ptitzler]

I don’t know about elyra-examples-kfp-catalog , is it safe to remove ?

Yes, absolutely. This optional package is installed by default, when you specify pip install elyra[all]. Since you are using the local runtime it’s functionality does not apply.

Error message from elyra alert:

Traceback (most recent call last):
  File "C:\Users\ufo\anaconda3\lib\site-packages\elyra\pipeline\local\processor_local.py", line 234, in process
    papermill.execute_notebook(filepath, filepath, **additional_kwargs)
  File "C:\Users\ufo\anaconda3\lib\site-packages\papermill\execute.py", line 107, in execute_notebook
    nb = papermill_engines.execute_notebook_with_engine(
  File "C:\Users\ufo\anaconda3\lib\site-packages\papermill\engines.py", line 49, in execute_notebook_with_engine
    return self.get_engine(engine_name).execute_notebook(nb, kernel_name, **kwargs)
  ...
  File "C:\Users\ufo\anaconda3\lib\site-packages\nbformat\v4\nbbase.py", line 68, in new_output
    validate(output, output_type)
  File "C:\Users\ufo\anaconda3\lib\site-packages\nbformat\v4\nbbase.py", line 40, in validate
    return validate_orig(node, ref=ref, version=nbformat)
  File "C:\Users\ufo\anaconda3\lib\site-packages\nbformat\validator.py", line 302, in validate
    raise error
nbformat.validator.NotebookValidationError: None is not of type 'array'

Failed validating 'type' in notebook['properties']['traceback']:

The stacktrace clarifies the following:

• the local runtime is used, which doesn’t support custom components (hence my earlier The only reference to the error message you've... question does not apply)
• the problem is caused by the 1.1.0 clean_data.ipynb Jupyter notebook

Two follow-up questions:

• When you open the notebook (after it was executed), does it contain any error messages? (Papermill inserts them)?
• Does the notebook work as-is when you run it in JupyterLab (Run > Run all cells)?