question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

ndarray TypeError when visualising Dask Array

See original GitHub issue

When trying to visualize a Dask array with raster and shade I get a TypeError: data must be an ndarray error in the eq_hist method. From the DataShader documentation I expected Dask arrays to be supported in DataShader.

I have provided a minimal notebook at https://gist.github.com/peterroelants/1d77e09bd05cc55c240bc11983e2a0c4 to reproduce the error.

ALL software version info

Python implementation: CPython
Python version       : 3.8.6
IPython version      : 7.19.0

Compiler    : GCC 7.5.0
OS          : Linux
Release     : 5.4.0-54-generic
Machine     : x86_64
Processor   : x86_64
CPU cores   : 4
Architecture: 64bit

datashader: 0.11.1
sys       : 3.8.6 | packaged by conda-forge | (default, Oct  7 2020, 19:08:05) 
[GCC 7.5.0]
numpy     : 1.19.4
xarray    : 0.16.1
dask      : 2.30.0

Description of expected behavior and the observed behavior

I expect shade being able to visualise a Dask array.

Complete, minimal, self-contained example code that reproduces the issue

https://gist.github.com/peterroelants/1d77e09bd05cc55c240bc11983e2a0c4

Stack traceback and/or browser JavaScript console output

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-6-a01bef0eda6c> in <module>
      1 canvas = datashader.Canvas(plot_width=900, plot_height=400)
      2 datashader.transfer_functions.Images(
----> 3     datashader.transfer_functions.shade(
      4         canvas.raster(data_da, agg=datashader.reductions.mean('z'))
      5     )

~/miniconda3/envs/msi/lib/python3.8/site-packages/datashader/transfer_functions/__init__.py in shade(agg, cmap, color_key, how, alpha, min_alpha, span, name, color_baseline)
    509 
    510     if agg.ndim == 2:
--> 511         return _interpolate(agg, cmap, how, alpha, span, min_alpha, name)
    512     elif agg.ndim == 3:
    513         return _colorize(agg, color_key, how, alpha, span, min_alpha, name, color_baseline)

~/miniconda3/envs/msi/lib/python3.8/site-packages/datashader/transfer_functions/__init__.py in _interpolate(agg, cmap, how, alpha, span, min_alpha, name)
    240     with np.errstate(invalid="ignore", divide="ignore"):
    241         # Transform data (log, eq_hist, etc.)
--> 242         data = interpolater(data, mask)
    243 
    244         # Transform span

~/miniconda3/envs/msi/lib/python3.8/site-packages/datashader/transfer_functions/__init__.py in eq_hist(data, mask, nbins)
    163         from._cuda_utils import interp
    164     elif not isinstance(data, np.ndarray):
--> 165         raise TypeError("data must be an ndarray")
    166     else:
    167         interp = np.interp

TypeError: data must be an ndarray

Where data is a dask.array.core.Array.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:8 (8 by maintainers)

github_iconTop GitHub Comments

1reaction
peterroelantscommented, Nov 30, 2020

QuadMesh should support Dask-backed Xarray quadmeshes properly since version 0.11.0 (see #885), and I don’t know of any regressions introduced in 0.11.1 or in master. So the first step would be to make a reproducible example of any bug or problem, and we can go from there. Thanks!

I created an example of what I meant and filed an issue at https://github.com/holoviz/datashader/issues/972 .

0reactions
jbednarcommented, Nov 29, 2020

QuadMesh should support Dask-backed Xarray quadmeshes properly since version 0.11.0 (see https://github.com/holoviz/datashader/pull/885), and I don’t know of any regressions introduced in 0.11.1 or in master. So the first step would be to make a reproducible example of any bug or problem, and we can go from there. Thanks!

Read more comments on GitHub >

github_iconTop Results From Across the Web

dask.array.register_chunk_type - Dask documentation
Array in the type casting hierarchy), a TypeError will be raised due to all operand types returning NotImplemented . Examples. Using a mock...
Read more >
dask.array.creation - Dask documentation
Returns ------- out : ndarray Array of ones with the same shape and type as `a`. ... Array): raise TypeError(f"v must be a...
Read more >
dask.array.core - Dask documentation
Array Register that a function implements the API of a NumPy function (or ... or: da.map_blocks(function, x, y, z)" ) raise TypeError(msg %...
Read more >
Source code for dask.array.routines
"See the numpy.histogram docstring for more information. ... {range.shape}" ) except TypeError: raise TypeError( f"Expected a sequence or array for range, ...
Read more >
Source code for dask.array.numpy_compat
import warnings import numpy as np from packaging.version import parse as ... np.divide ma_divide = np.ma.divide except TypeError: # Divide with dtype ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found