Badly formed notebook tracebacks ending with `AssertionError: Pieces mismatches`

We’ve been seeing this more and more lately. Sometimes when an error is raised internally by HoloViews the handling of the traceback itself raises another error, making the traceback pretty difficult to parse. Here’s an example to reproduce that.

import hvplot.pandas  # noqa
import pandas as pd
import random

def gen_df():
    df = pd.DataFrame({
        'month': dates.month,
        'hour': dates.hour,
        'day': dates.day,
        'value': [random.randint(0, 100) for i in range(len(dates))],
    })
    
    df['id'] = 'test'
    
    return df
    

dates = pd.date_range("2022-09-01", "2022-10-31", freq='h')

df = gen_df()

for i in range(3):
    df = pd.concat([df, gen_df()])

df.hvplot.heatmap(x='hour', y='day', C='id', groupby='month', reduce_function='count')

In a classic notebook the output traceback is preceded by Unexpected exception formatting exception. Falling back to standard exception.

Full traceback:

raceback (most recent call last):
  File "/Users/mliquet/miniconda3/envs/hv-dev38/lib/python3.8/site-packages/IPython/core/formatters.py", line 222, in catch_format_error
    r = method(self, *args, **kwargs)
  File "/Users/mliquet/miniconda3/envs/hv-dev38/lib/python3.8/site-packages/IPython/core/formatters.py", line 973, in __call__
    return method(include=include, exclude=exclude)
  File "/Users/mliquet/work/dev/holoviews/holoviews/core/dimension.py", line 1293, in _repr_mimebundle_
    return Store.render(self)
  File "/Users/mliquet/work/dev/holoviews/holoviews/core/options.py", line 1427, in render
    ret = hook(obj)
  File "/Users/mliquet/work/dev/holoviews/holoviews/ipython/display_hooks.py", line 277, in pprint_display
    return display(obj, raw_output=True)
  File "/Users/mliquet/work/dev/holoviews/holoviews/ipython/display_hooks.py", line 253, in display
    output = map_display(obj)
  File "/Users/mliquet/work/dev/holoviews/holoviews/ipython/display_hooks.py", line 141, in wrapped
    mimebundle = fn(element, max_frames=max_frames)
  File "/Users/mliquet/work/dev/holoviews/holoviews/ipython/display_hooks.py", line 201, in map_display
    return render(vmap)
  File "/Users/mliquet/work/dev/holoviews/holoviews/ipython/display_hooks.py", line 68, in render
    return renderer.components(obj, **kwargs)
  File "/Users/mliquet/work/dev/holoviews/holoviews/plotting/renderer.py", line 398, in components
    return self._render_panel(plot, embed, comm)
  File "/Users/mliquet/work/dev/holoviews/holoviews/plotting/renderer.py", line 405, in _render_panel
    model = plot.layout._render_model(doc, comm)
  File "/Users/mliquet/miniconda3/envs/hv-dev38/lib/python3.8/site-packages/panel/viewable.py", line 505, in _render_model
    model = self.get_root(doc, comm)
  File "/Users/mliquet/miniconda3/envs/hv-dev38/lib/python3.8/site-packages/panel/viewable.py", line 556, in get_root
    root = self._get_model(doc, comm=comm)
  File "/Users/mliquet/miniconda3/envs/hv-dev38/lib/python3.8/site-packages/panel/layout/base.py", line 146, in _get_model
    objects = self._get_objects(model, [], doc, root, comm)
  File "/Users/mliquet/miniconda3/envs/hv-dev38/lib/python3.8/site-packages/panel/layout/base.py", line 131, in _get_objects
    child = pane._get_model(doc, root, model, comm)
  File "/Users/mliquet/miniconda3/envs/hv-dev38/lib/python3.8/site-packages/panel/pane/holoviews.py", line 265, in _get_model
    plot = self._render(doc, comm, root)
  File "/Users/mliquet/miniconda3/envs/hv-dev38/lib/python3.8/site-packages/panel/pane/holoviews.py", line 342, in _render
    return renderer.get_plot(self.object, **kwargs)
  File "/Users/mliquet/work/dev/holoviews/holoviews/plotting/bokeh/renderer.py", line 70, in get_plot
    plot = super().get_plot(obj, doc, renderer, **kwargs)
  File "/Users/mliquet/work/dev/holoviews/holoviews/plotting/renderer.py", line 240, in get_plot
    plot.update(init_key)
  File "/Users/mliquet/work/dev/holoviews/holoviews/plotting/plot.py", line 948, in update
    return self.initialize_plot()
  File "/Users/mliquet/work/dev/holoviews/holoviews/plotting/bokeh/element.py", line 1380, in initialize_plot
    ranges = self.compute_ranges(self.hmap, key, ranges)
  File "/Users/mliquet/work/dev/holoviews/holoviews/plotting/plot.py", line 607, in compute_ranges
    self._compute_group_range(group, elements, ranges, framewise,
  File "/Users/mliquet/work/dev/holoviews/holoviews/plotting/plot.py", line 717, in _compute_group_range
    data_range = el.range(el_dim, dimension_range=False)
  File "/Users/mliquet/work/dev/holoviews/holoviews/core/data/__init__.py", line 204, in pipelined_fn
    result = method_fn(*args, **kwargs)
  File "/Users/mliquet/work/dev/holoviews/holoviews/element/raster.py", line 853, in range
    self.gridded._binned = False
  File "/Users/mliquet/work/dev/holoviews/holoviews/element/raster.py", line 819, in gridded
    self._gridded = categorical_aggregate2d(self)
  File "/Users/mliquet/miniconda3/envs/hv-dev38/lib/python3.8/site-packages/param/parameterized.py", line 3654, in __new__
    return inst.__call__(*args,**params)
  File "/Users/mliquet/work/dev/holoviews/holoviews/core/operation.py", line 220, in __call__
    return element.apply(self, **kwargs)
  File "/Users/mliquet/work/dev/holoviews/holoviews/core/accessors.py", line 45, in pipelined_call
    result = __call__(*args, **kwargs)
  File "/Users/mliquet/work/dev/holoviews/holoviews/core/accessors.py", line 202, in __call__
    new_obj = apply_function(self._obj, **inner_kwargs)
  File "/Users/mliquet/work/dev/holoviews/holoviews/core/operation.py", line 214, in __call__
    return self._apply(element)
  File "/Users/mliquet/work/dev/holoviews/holoviews/core/operation.py", line 141, in _apply
    ret = self._process(element, key)
  File "/Users/mliquet/work/dev/holoviews/holoviews/element/util.py", line 229, in _process
    raise ValueError("Must have at two dimensions to aggregate over"
ValueError: Must have at two dimensions to aggregate overand one value dimension to aggregate on.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/mliquet/miniconda3/envs/hv-dev38/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 1997, in showtraceback
    stb = self.InteractiveTB.structured_traceback(
  File "/Users/mliquet/miniconda3/envs/hv-dev38/lib/python3.8/site-packages/IPython/core/ultratb.py", line 1112, in structured_traceback
    return FormattedTB.structured_traceback(
  File "/Users/mliquet/miniconda3/envs/hv-dev38/lib/python3.8/site-packages/IPython/core/ultratb.py", line 1006, in structured_traceback
    return VerboseTB.structured_traceback(
  File "/Users/mliquet/miniconda3/envs/hv-dev38/lib/python3.8/site-packages/IPython/core/ultratb.py", line 859, in structured_traceback
    formatted_exception = self.format_exception_as_a_whole(etype, evalue, etb, number_of_lines_of_context,
  File "/Users/mliquet/miniconda3/envs/hv-dev38/lib/python3.8/site-packages/IPython/core/ultratb.py", line 812, in format_exception_as_a_whole
    frames.append(self.format_record(r))
  File "/Users/mliquet/miniconda3/envs/hv-dev38/lib/python3.8/site-packages/IPython/core/ultratb.py", line 730, in format_record
    result += ''.join(_format_traceback_lines(frame_info.lines, Colors, self.has_colors, lvals))
  File "/Users/mliquet/miniconda3/envs/hv-dev38/lib/python3.8/site-packages/stack_data/utils.py", line 145, in cached_property_wrapper
    value = obj.__dict__[self.func.__name__] = self.func(obj)
  File "/Users/mliquet/miniconda3/envs/hv-dev38/lib/python3.8/site-packages/stack_data/core.py", line 734, in lines
    pieces = self.included_pieces
  File "/Users/mliquet/miniconda3/envs/hv-dev38/lib/python3.8/site-packages/stack_data/utils.py", line 145, in cached_property_wrapper
    value = obj.__dict__[self.func.__name__] = self.func(obj)
  File "/Users/mliquet/miniconda3/envs/hv-dev38/lib/python3.8/site-packages/stack_data/core.py", line 677, in included_pieces
    scope_pieces = self.scope_pieces
  File "/Users/mliquet/miniconda3/envs/hv-dev38/lib/python3.8/site-packages/stack_data/utils.py", line 145, in cached_property_wrapper
    value = obj.__dict__[self.func.__name__] = self.func(obj)
  File "/Users/mliquet/miniconda3/envs/hv-dev38/lib/python3.8/site-packages/stack_data/core.py", line 617, in scope_pieces
    for piece in self.source.pieces
  File "/Users/mliquet/miniconda3/envs/hv-dev38/lib/python3.8/site-packages/stack_data/utils.py", line 145, in cached_property_wrapper
    value = obj.__dict__[self.func.__name__] = self.func(obj)
  File "/Users/mliquet/miniconda3/envs/hv-dev38/lib/python3.8/site-packages/stack_data/core.py", line 106, in pieces
    return list(self._clean_pieces())
  File "/Users/mliquet/miniconda3/envs/hv-dev38/lib/python3.8/site-packages/stack_data/core.py", line 130, in _clean_pieces
    raise AssertionError("Pieces mismatches: %s" % mismatches)
AssertionError: Pieces mismatches: [{680, 681}, {696, 695}, {708, 709}, {714, 715}]

HoloViews is fiddling with the traceback with AbbreviatedException, that might be something to look into. I’ve also found this Ipython issue that seems relevant https://github.com/ipython/ipython/issues/13598 and would indicate some kind of issue/incompatibility in Ipython 8 (8.0.0 was released in January 2022).

cc @Hoxbro

[maximlt]

Damned, I think I found it. Something seems to be very unhappy with how this is formatted in the param.parameterized module:

PInfo = namedtuple("PInfo", "inst cls name pobj what"); _add_doc(PInfo,
    """
    Object describing something being watched about a Parameter.

    `inst`: Parameterized instance owning the Parameter, or None

    `cls`: Parameterized class owning the Parameter

    `name`: Name of the Parameter being watched

    `pobj`: Parameter object being watched

    `what`: What is being watched on the Parameter (either 'value' or a slot name)
    """)

Particularly about the multiple statements in a line with ; and continuing the last statement over multiple lines. I can reproduce by importing this module in a Jupyter Notebook:

# reproduce.py
raise ValueError('error')

a = 1 ; b = ("""
   unhappy """
)

I’m going to open an issue on IPython’s issue tracker, and make a change to param.parameterized not to get this error there (there may be other places but most HoloViz error tracebacks traverse this file).

[maximlt]

This has been fixed in stack_data 0.6.2 (and a workaround implemented in Param https://github.com/holoviz/param/pull/663).