question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

concatenate zarr datasets

See original GitHub issue

I would like to virtually concatenate a few zarr arrays along an axis. They are specifically not xarray datasets, and do not have coordinate arrays.

Is this in scope for kerchunks API?

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:11 (6 by maintainers)

github_iconTop GitHub Comments

3reactions
martindurantcommented, Sep 1, 2022

I may attempt it today, if it looks like it may be as simple as I assume.

0reactions
ivirshupcommented, Sep 1, 2022

I would be happy to see a simplified combine/concat like you suggest. Except, the inputs are not zarr arrays but reference sets (JSON or dict);

Practically, how would you see this happening? Would you or someone already working on the project be interested in implementing this, or is this more “PRs welcome”?

I could try, but can’t promise any results, since I’m still mainly hitting errors with MultiZarrToZarr and have no experience with fsspec 😅

Read more comments on GitHub >

github_iconTop Results From Across the Web

Tutorial — zarr 2.13.3 documentation - Read the Docs
Zarr provides classes and functions for working with N-dimensional arrays that behave like NumPy arrays but whose data is divided into chunks and...
Read more >
Why does concatenating zarr.ZipStores with xarray and dask ...
I often want to concatenate sets of zarr collections (stored on a file system) to write back out as single dataset.
Read more >
Concatenating datasets with variable length strings ... - GitHub
I want to concatenate a collection of datasets with a variable that contains variable length strings and save them to Zarr. The strings...
Read more >
xarray.Dataset.to_zarr
xarray.Dataset.to_zarr# · From the chunks attribute in each variable's encoding (can be set via Dataset. · If the variable is a Dask array,...
Read more >
How to use Dask.Array.From_Zarr to open a zarr file on Dask?
fp = os.path.join( ...: ROOT_DIR, . ... ds = xr.open_zarr(fp) . ... When you read in a zarr dataset, the result will be...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found