question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Zarr custom Reader?

See original GitHub issue

with the release of GDAL 3.4, the Zarr driver is now available by default, which mean that rasterio/rio-tiler can now open/read Zarr files.

Why do we need a custom reader

  • Zarr format can be quite different from a COG or other raster format because it’s not restricted to 2D dataset.

If the Zarr dataset contains one single array with 2 dimensions, it will be exposed as a regular GDALDataset when using the classic raster API. If the dataset contains more than one such single array, or arrays with 3 or more dimensions, the driver will list subdatasets to access each array and/or 2D slices within arrays with 3 or more dimensions.

ref: https://gdal.org/drivers/raster/zarr.html#particularities-of-the-classic-raster-api

  • No Overviews

Because there is no overview (currently in dev) we cannot fetch lower resolution of the data and should restrict to one zoom level

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:6 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
sgilliescommented, Oct 28, 2022

@vincentsarago I’m giving up on hierarchical data in rasterio. See https://github.com/rasterio/rasterio/issues/1759#issuecomment-1294298813. It seems like a lot of wasted effort on redundant Python software. I know I have flip-flopped on this in the past, but now I believe that using zarr (python) or h5py is the way to go.

0reactions
vincentsaragocommented, Oct 28, 2022

thanks for trying @sgillies. I commented on the issue directly.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Tutorial — zarr 2.13.3 documentation - Read the Docs
Zarr arrays support a similar interface to NumPy arrays for reading and writing ... Here is an example using LZMA with a custom...
Read more >
xarray Internals
xarray can wrap custom duck array objects as long as they define numpy's shape ... When reading a Zarr group, Xarray looks for...
Read more >
Reading and writing files - Xarray
Xarray's Zarr backend allows xarray to leverage these capabilities, including the ability to store and analyze datasets far too large fit onto ...
Read more >
Zarr files and L5kit data for dummies - Kaggle
Zarr arrays support a similar interface to NumPy arrays for reading and writing data. ... Zarr arrays and groups support custom key/value attributes, ......
Read more >
Create Dask Arrays - Dask documentation
Load array from the zarr storage format ... it is ideal for parallel access in both reading and writing (for the latter, ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found