question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Improved xarray support

See original GitHub issue

I noticed some points concerning xarray support in holoviews which could be improved:

  • Both xarray and holoviews support measurements units, but it appears that they are lost during in- and exporting. I think it would be pretty straightforward to add this feature in the holoviews importing and exporting code, I could look into it.

  • When importing a netcdf file via xarray and passing it to hv.Dataset, the key dimensions are sometimes ordered differently, depending whether the file is imported as a xr.DataArrayor xr.Dataset. This is not a gigantic problem, especially since one can manually specify the key dimensions’ order, although it is still pretty annoying when dealing with images… EDIT: (actually, this seems to have been fixed already sometime between 1.9.3 and the current master)

  • I’ve never had any luck converting the data storage backend from ‘xarray’ to anything else, writing img.clone(datatype=['image']), as shown in the gridded datasets guide: http://holoviews.org/user_guide/Gridded_Datasets.html Perhaps it would be nice to show this more explicitly in the documentation, or fix it if it is a bug.

EDIT:

  • I’ve noticed that dimensions without coordinates are not accepted by XArrayInterface - although omitting coordinates seems to be the recommended way in xarray to deal with dimensions without meaningful coordinates (such as pixels in an image). In this case, it should be possible to just automatically assign simple range coordinates to these dimensions when making a holoviews Dataset.

  • The treatment of non-dimension coordinates could be more concise. Take for example the “rasm” example that comes with xarray, which looks like this

Coordinates:
  * time     (time) datetime64[ns] 1980-09-16T12:00:00 1980-10-17 ...
    xc       (y, x) float64 189.2 189.4 189.6 189.7 189.9 190.1 190.2 190.4 ...
    yc       (y, x) float64 16.53 16.78 17.02 17.27 17.51 17.76 18.0 18.25 ...
Dimensions without coordinates: x, y
Data variables:
    Tair     (time, y, x) float64 nan nan nan nan nan nan nan nan nan nan ...

Right now, coordinates are converted into kdims, and dimensions without coordinates are ignored. IMO, it would be better to let the user select if they want (time, y, c) or (time, yc, xc) coordinates when making the holoviews Dataset, where the default should be the proper dimensions.

  • The xarray interface in holoviews could also benefit from more documentation in general - i.e. an example of how to load an example file from xarray into holoviews and make a plot. Introducing xarray support by cloning from a different datatype does not seem very intuitive to me.

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Reactions:1
  • Comments:23 (23 by maintainers)

github_iconTop GitHub Comments

1reaction
drs251commented, Apr 3, 2018

It seems like your latest changes to xarray overwrite existing units on the dimensions.

That’s a point I did not think about previously (I’m still using 1.9.5, not sure if it’s a good idea to use the current master for everyday work). Ideally, specified key and value dimensions should override anything thats present in the xarray, but I haven’t checked if that’s the case - I’ll look into it, perhaps another PR is needed.

Concrete examples would be appreciated, I’m not currently aware of any issues with .redim atm.

xdim = hv.Dimension("x", label="my x dimension", unit="mJ")
ydim = hv.Dimension("y", label="my y dimension", unit="kB")
zdim = hv.Dimension("z", label="my z dimension", unit="fs")
image = hv.Dataset(([0, 1], [1, 2], [[3, 4], [5, 6]]), kdims=[xdim, ydim], vdims=[zdim]).to(hv.Image)
image = image.redim(z='time')
image = image.redim.unit(x='eV', y='TB')
image = image.redim.label(x='Energy', y='Capacity')

Actually, I saw that my only issue with redim is that it refuses to change existing labels - everything else works like I would expect it to. So as far as I know, if you come across a dimension label you don’t like, the only option is to extract the dimension values and rebuild the element from scratch. I would prefer if redim worked the same way on labels as it does on units and dimension names.

0reactions
philippjfrcommented, Jun 8, 2020

I think all the issues here were addressed, if I’m mistaken please reopen a more specific issue.

Read more comments on GitHub >

github_iconTop Results From Across the Web

What's New - Xarray
This release brings improved support for pint arrays, methods for weighted standard deviation, variance, and sum of squares, the option to disable the...
Read more >
Improved xarray support #2319 - holoviz/holoviews - GitHub
I noticed some points concerning xarray support in holoviews which could be improved: Both xarray and holoviews support measurements units, ...
Read more >
What's New — xarray 0.11.0 documentation - PyData |
Xarray plans to drop support for python 2.7 at the end of 2018. ... For improved usability and consistency with pandas, in the...
Read more >
Indexing and selecting data - xarray - Read the Docs
Thus in total, xarray supports four different kinds of indexing, ... You may find increased performance by loading your data into memory first,...
Read more >
xarray - Google Groups
Discussion group for xarray: N-D labeled arrays and datasets in Python ... This release brings improved support for pint arrays, methods for. unread,....
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found