hvplot is not using lazy loading for zarr-backed xarray dataset
See original GitHub issueALL software version info
xr.__version__
'0.21.1'
hvplot.__version__
'0.7.3'
holoviews.__version__
'1.14.7'
Description of expected behavior and the observed behavior
When I visualize zarr-backed xarray datasets with hvplot, I expect the loading to be lazy. I don’t want to load the entire array into memory. I expect this to happen even without dask being involved.
Instead, I am noticing that hvplot is loading data eagerly.
Complete, minimal, self-contained example code that reproduces the issue
import xarray as xr
import hvplot.xarray
url = "https://ncsa.osn.xsede.org/Pangeo/pangeo-forge/swot_adac/FESOM/surf/fma.zarr"
ds = xr.open_dataset(url, engine='zarr') # note that ds is not chunked but still uses lazy loading
# this will trigger loading the whole array into memory
ds.sst.hvplot(x='lon', y='lat', dynamic=True, rasterize=True)
Note that lazyness can be achieved by chunking the xarray dataset. But I don’t want to have to do that. Dask introduces additional latency that I would like to avoid here.
Issue Analytics
- State:
- Created 2 years ago
- Comments:19 (12 by maintainers)
Top Results From Across the Web
Gridded Data — hvPlot 0.8.2 documentation
This user guide will cover how to leverage xarray and hvplot to visualize and explore data of different dimensionality ranging from simple 1D...
Read more >Can I force xarray.open_dataset() to do a lazy load?
A colleague of mine gave me the answer: I needed to pass the option decode_times=False . With this option, xarray.open_dataset() does lazy ......
Read more >hvplot - bytemeta
Odd behavior with invert_yaxis, invert_xaxis · hvplot is not using lazy loading for zarr-backed xarray dataset.
Read more >Gprof2dot is not recognized as an internal or external ...
1 dev BSD-3-Clause X Machine learning with dirty categories. · 2 MIT X X X Python 2.7 backport of the "dis" module from...
Read more >A high-level plotting API for pandas, dask, xarray, and ...
When I visualize zarr-backed xarray datasets with hvplot, I expect the loading to be lazy. I don't want to load the entire array...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
I’m on xarray 2022.3.0 and this is sufficient to reproduce:
There is a time dimension.
will lazily grab a single time step, which is all that should be required for a single image plot.