Automatic chunking of arrays ?

Hi there,

Hopefully this turns out to be a basic issue, but I was wondering why the chunks='auto' that dask seems to provide (https://docs.dask.org/en/latest/array-chunks.html#automatic-chunking) isn’t an option for xarray? I’m not 100% sure of how dask decides how to automatically chunk its arrays, so maybe there’s a compatibility issue?

I get the impression that the dask method automatically tries to prevent the issues of “too many chunks” or “too few chunks” which can sometimes happen when choosing chunk sizes automatically. If so, it would maybe be a useful thing to include in future versions?

Happy to be corrected if I’ve misunderstood something here though, still getting my head around how the dask/xarray compatibility really works…

Cheers!

[shoyer]

If we think can improve an error message by adding additional context, the right solution is to use raise Exception(...) from original_error: https://stackoverflow.com/a/16414892/809705

On the other hand, if xarray doesn’t have anything more to add on top of the original error message, it is best not to add any wrapper at all. Users will just see the original error from dask.

[shoyer]

The error message from dask is already pretty descriptive: NotImplementedError: Can not use auto rechunking with object dtype. We are unable to estimate the size in bytes of object data

I don’t think we have much to add on top of that?