dask.array.asarray should recognize dask duck-types

With master for dask and xarray (with dask duck array support):

In [25]: import dask.array as da

In [26]: import xarray

In [27]: dask_obj = da.ones((3, 3), chunks=-1)

In [28]: xarray_obj = xarray.DataArray(x, dims=['x', 'y'])

In [29]: dict(da.asarray(xarray_obj).dask)
Out[29]:
{('array-5ce95599636c17df4c5746fb5f3c220b',
  0,
  0): (<function dask.array.core.getter_inline>, 'array-original-5ce95599636c17df4c5746fb5f3c220b', (slice(0, 3, None),
   slice(0, 3, None))),
 'array-original-5ce95599636c17df4c5746fb5f3c220b': <xarray.DataArray 'wrapped-8015d128b19387281f3d8bd3a663300a' (x: 3, y: 3)>
 dask.array<shape=(3, 3), dtype=float64, chunksize=(3, 3)>
 Dimensions without coordinates: x, y}

The dask graph from asarray() has an xarray.DataArray object in it, with its own nested dask graph. This is pretty messy: asarray() should really recognize the dask duck-type and combine the argument’s dask graph into its own.

[shoyer]

Since xarray.DataArray does not inherit from dask Array, both asarray and asanyarray should convert it in a dask Array. That part of the existing functionality is exactly right.

But if a xarray.DataArray contains a dask graph (via the custom collections interface), it would be nice to merge its dask graph directly into the resulting dask Array, instead of having nested dask arrays. The later is pretty messy, and can result in various problems like the inability to optimize and resource contention.

It’s not entirely clear to me that there is a good way to do this with the current custom collections interface. @mrocklin @jcrist any thoughts?

[shoyer]

xarray.DataArray does support the custom collections interface.

But merely being a dask object isn’t enough here. We also need some way to indicate how xarray objects can be converted into dask arrays.

Coincidentally I’ve been working this past week with @njsmith on a proposal for how exactly this sort of thing should be done. We’ll hopefully be sending it out very soon…