Dynspread fails to handle NaN properly

ALL software version info

HoloViews: 1.13.2 Pandas: 0.24.2 NumPy: 1.18.1 Dask: 2.14.0

Description of expected behavior and the observed behavior

Calling the function dynspread(datashade(x)), where x is an element that gets correctly displayed just by calling datashade (e.g., a hv.Curve element) yields ZeroDivisionError if x is a vector full of NaN.

Screenshots or screencasts of the bug in action

Stack traceback and/or browser JavaScript console output

---------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
C:\Anaconda3\lib\site-packages\IPython\core\formatters.py in __call__(self, obj, include, exclude)
    968 
    969             if method is not None:
--> 970                 return method(include=include, exclude=exclude)
    971             return None
    972         else:

C:\Anaconda3\lib\site-packages\holoviews\core\dimension.py in _repr_mimebundle_(self, include, exclude)
   1302         combined and returned.
   1303         """
-> 1304         return Store.render(self)
   1305 
   1306 

C:\Anaconda3\lib\site-packages\holoviews\core\options.py in render(cls, obj)
   1393         data, metadata = {}, {}
   1394         for hook in hooks:
-> 1395             ret = hook(obj)
   1396             if ret is None:
   1397                 continue

C:\Anaconda3\lib\site-packages\holoviews\ipython\display_hooks.py in pprint_display(obj)
    280     if not ip.display_formatter.formatters['text/plain'].pprint:
    281         return None
--> 282     return display(obj, raw_output=True)
    283 
    284 

C:\Anaconda3\lib\site-packages\holoviews\ipython\display_hooks.py in display(obj, raw_output, **kwargs)
    253     elif isinstance(obj, (Layout, NdLayout, AdjointLayout)):
    254         with option_state(obj):
--> 255             output = layout_display(obj)
    256     elif isinstance(obj, (HoloMap, DynamicMap)):
    257         with option_state(obj):

C:\Anaconda3\lib\site-packages\holoviews\ipython\display_hooks.py in wrapped(element)
    144         try:
    145             max_frames = OutputSettings.options['max_frames']
--> 146             mimebundle = fn(element, max_frames=max_frames)
    147             if mimebundle is None:
    148                 return {}, {}

C:\Anaconda3\lib\site-packages\holoviews\ipython\display_hooks.py in layout_display(layout, max_frames)
    218         return None
    219 
--> 220     return render(layout)
    221 
    222 

C:\Anaconda3\lib\site-packages\holoviews\ipython\display_hooks.py in render(obj, **kwargs)
     66         renderer = renderer.instance(fig='png')
     67 
---> 68     return renderer.components(obj, **kwargs)
     69 
     70 

C:\Anaconda3\lib\site-packages\holoviews\plotting\renderer.py in components(self, obj, fmt, comm, **kwargs)
    374             doc = Document()
    375             with config.set(embed=embed):
--> 376                 model = plot.layout._render_model(doc, comm)
    377             if embed:
    378                 return render_model(model, comm)

C:\Anaconda3\lib\site-packages\panel\viewable.py in _render_model(self, doc, comm)
    363         if comm is None:
    364             comm = state._comm_manager.get_server_comm()
--> 365         model = self.get_root(doc, comm)
    366 
    367         if config.embed:

C:\Anaconda3\lib\site-packages\panel\viewable.py in get_root(self, doc, comm)
    534         """
    535         doc = doc or _curdoc()
--> 536         root = self._get_model(doc, comm=comm)
    537         self._preprocess(root)
    538         ref = root.ref['id']

C:\Anaconda3\lib\site-packages\panel\layout.py in _get_model(self, doc, root, parent, comm)
    134         if root is None:
    135             root = model
--> 136         objects = self._get_objects(model, [], doc, root, comm)
    137         props = dict(self._init_properties(), objects=objects)
    138         model.update(**self._process_param_change(props))

C:\Anaconda3\lib\site-packages\panel\layout.py in _get_objects(self, model, old_objects, doc, root, comm)
    124             else:
    125                 try:
--> 126                     child = pane._get_model(doc, root, model, comm)
    127                 except RerenderError:
    128                     return self._get_objects(model, current_objects[:i], doc, root, comm)

C:\Anaconda3\lib\site-packages\panel\pane\holoviews.py in _get_model(self, doc, root, parent, comm)
    225             plot = self.object
    226         else:
--> 227             plot = self._render(doc, comm, root)
    228 
    229         plot.pane = self

C:\Anaconda3\lib\site-packages\panel\pane\holoviews.py in _render(self, doc, comm, root)
    284             kwargs = {}
    285 
--> 286         return renderer.get_plot(self.object, **kwargs)
    287 
    288     def _cleanup(self, root):

C:\Anaconda3\lib\site-packages\holoviews\plotting\bokeh\renderer.py in get_plot(self_or_cls, obj, doc, renderer, **kwargs)
     71         combining the bokeh model with another plot.
     72         """
---> 73         plot = super(BokehRenderer, self_or_cls).get_plot(obj, doc, renderer, **kwargs)
     74         if plot.document is None:
     75             plot.document = Document() if self_or_cls.notebook_context else curdoc()

C:\Anaconda3\lib\site-packages\holoviews\plotting\renderer.py in get_plot(self_or_cls, obj, doc, renderer, comm, **kwargs)
    209 
    210         # Initialize DynamicMaps with first data item
--> 211         initialize_dynamic(obj)
    212 
    213         if not renderer:

C:\Anaconda3\lib\site-packages\holoviews\plotting\util.py in initialize_dynamic(obj)
    249             continue
    250         if not len(dmap):
--> 251             dmap[dmap._initial_key()]
    252 
    253 

C:\Anaconda3\lib\site-packages\holoviews\core\spaces.py in __getitem__(self, key)
   1278         # Not a cross product and nothing cached so compute element.
   1279         if cache is not None: return cache
-> 1280         val = self._execute_callback(*tuple_key)
   1281         if data_slice:
   1282             val = self._dataslice(val, data_slice)

C:\Anaconda3\lib\site-packages\holoviews\core\spaces.py in _execute_callback(self, *args)
   1052 
   1053         with dynamicmap_memoization(self.callback, self.streams):
-> 1054             retval = self.callback(*args, **kwargs)
   1055         return self._style(retval)
   1056 

C:\Anaconda3\lib\site-packages\holoviews\core\spaces.py in __call__(self, *args, **kwargs)
    660         kwarg_hash = kwargs.pop('_memoization_hash_', ())
    661         (self.args, self.kwargs) = (args, kwargs)
--> 662         if not args and not kwargs and not any(kwarg_hash): return self.callable()
    663         inputs = [i for i in self.inputs if isinstance(i, DynamicMap)]
    664         streams = []

C:\Anaconda3\lib\site-packages\holoviews\util\__init__.py in dynamic_operation(*key, **kwargs)
    984         def dynamic_operation(*key, **kwargs):
    985             key, obj = resolve(key, kwargs)
--> 986             return apply(obj, *key, **kwargs)
    987 
    988         operation = self.p.operation

C:\Anaconda3\lib\site-packages\holoviews\util\__init__.py in apply(element, *key, **kwargs)
    980         def apply(element, *key, **kwargs):
    981             kwargs = dict(util.resolve_dependent_kwargs(self.p.kwargs), **kwargs)
--> 982             return self._process(element, key, kwargs)
    983 
    984         def dynamic_operation(*key, **kwargs):

C:\Anaconda3\lib\site-packages\holoviews\util\__init__.py in _process(self, element, key, kwargs)
    962         elif isinstance(self.p.operation, Operation):
    963             kwargs = {k: v for k, v in kwargs.items() if k in self.p.operation.param}
--> 964             return self.p.operation.process_element(element, key, **kwargs)
    965         else:
    966             return self.p.operation(element, **kwargs)

C:\Anaconda3\lib\site-packages\holoviews\core\operation.py in process_element(self, element, key, **params)
    170             self.p = param.ParamOverrides(self, params,
    171                                           allow_extra_keywords=self._allow_extra_keywords)
--> 172         return self._apply(element, key)
    173 
    174 

C:\Anaconda3\lib\site-packages\holoviews\core\operation.py in _apply(self, element, key)
    130         element_pipeline = getattr(element, '_pipeline', None)
    131 
--> 132         ret = self._process(element, key)
    133         for hook in self._postprocess_hooks:
    134             ret = hook(self, ret, **kwargs)

C:\Anaconda3\lib\site-packages\holoviews\operation\datashader.py in _process(self, element, key)
   1557         else:
   1558             new_data = array
-> 1559         return element.clone(new_data, **kwargs)
   1560 
   1561 

C:\Anaconda3\lib\site-packages\holoviews\element\raster.py in clone(self, data, shared_data, new_type, link, *args, **overrides)
    427             overrides = dict(sheet_params, **overrides)
    428         return super(Image, self).clone(data, shared_data, new_type, link,
--> 429                                         *args, **overrides)
    430 
    431 

C:\Anaconda3\lib\site-packages\holoviews\core\data\__init__.py in clone(self, data, shared_data, new_type, link, *args, **overrides)
   1155 
   1156         new_dataset = super(Dataset, self).clone(
-> 1157             data, shared_data, new_type, *args, **overrides
   1158         )
   1159 

C:\Anaconda3\lib\site-packages\holoviews\core\dimension.py in clone(self, data, shared_data, new_type, link, *args, **overrides)
    569         # Apply name mangling for __ attribute
    570         pos_args = getattr(self, '_' + type(self).__name__ + '__pos_params', [])
--> 571         return clone_type(data, *args, **{k:v for k,v in settings.items()
    572                                           if k not in pos_args})
    573 

C:\Anaconda3\lib\site-packages\holoviews\element\raster.py in __init__(self, data, kdims, vdims, **params)
    676             # Handle all forms of packed value dimensions
    677             vdims.append(alpha)
--> 678         super(RGB, self).__init__(data, kdims=kdims, vdims=vdims, **params)
    679 
    680 

C:\Anaconda3\lib\site-packages\holoviews\element\raster.py in __init__(self, data, kdims, vdims, bounds, extents, xdensity, ydensity, rtol, **params)
    322             l, b, r, t = bounds.lbrt()
    323             xdensity = xdensity if xdensity else util.compute_density(l, r, dim1, self._time_unit)
--> 324             ydensity = ydensity if ydensity else util.compute_density(b, t, dim2, self._time_unit)
    325         SheetCoordinateSystem.__init__(self, bounds, xdensity, ydensity)
    326         if non_finite:

C:\Anaconda3\lib\site-packages\holoviews\core\util.py in compute_density(start, end, length, time_unit)
   1991         return (length/(diff.total_seconds()*tscale))
   1992     else:
-> 1993         return length/diff
   1994 
   1995 

ZeroDivisionError: float division by zero

[jbednar]

A PR would be gratefully accepted, but I’m not requesting one because I’m not yet sure precisely where it’s most appropriate to make this change. I do think it’s a situation we already dealt with earlier for all-zero plots, so a quick check of the codebase might find where we should simply consider all-NaN to be similar to all-zero.

We’re already planning to improve the legend handling for Datashader plots in HoloViews, mostly by re-implementing the features of the shade operation in JavaScript so that Bokeh has access to all the data it needs for building colorbars and legends. You might find https://anaconda.org/jbednar/datashade_vs_rasterize/notebook helpful in the meantime.

[jlstevens]

Looking at the issue description and traceback, I am rather puzzled. The output of datashade(x) is shown to work regardless of NaNs and datashade should map to RGB(A) space where NaN is a transparent pixel. By the time this gets passed to dynspread (and internal to datashader, spread) all the NaNs should be gone and the datatype should be uint32.

Inside datashader.py, the error occurs with return element.clone(new_data, **kwargs) where element is the input element to the operation (which displays fine, as RGB) and the final error is down to how the x and y densities (i.e the spatial configuration, not the array values!) are computed, resulting in a zero division. My only theory is that the xarray coordinates of new_data have been somehow messed up even though an operation such as spread should never change them from their original values. I’ll now check my hunch by investigating further!