question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

API design for pointwise indexing

See original GitHub issue

There have been a number of threads discussing possible improvements/extensions to xray indexing. The current indexing behavior for isel is orthogonal indexing - in other words, each coordinate is treated independently (see #214 and #411 for more discussion).

So the question: what is the best way to incorporate diagonal or pointwise indexing in xray? I see two main goals / applications:

  1. support simple form of numpy style integer array indexing
  2. support pointwise array indexing along coordinates via computation of nearest-neighbor indexes - I think this can also be thought of as a form of resampling.

Input from @WeatherGod, @wholmgren, and @shoyer would be great.

Issue Analytics

  • State:open
  • Created 8 years ago
  • Comments:38 (33 by maintainers)

github_iconTop GitHub Comments

2reactions
duncanwpcommented, Jan 9, 2018

Further to the comment I made in a related issue #486 comment I’ve now taken a simplified version of the collocation approach in CIS and created a stand-alone package which works with xarray objects: https://github.com/cistools/collocate.

This works essentially the same as the nice example shown in the above blog, with some key differences:

  • The points within a certain distance (tolerance) of each sample point can be aggregated or selected from using the built-in kernels, allowing fast operations over many sample points.
  • The horizontal distance constraint can be supplemented with constraints in other dimensions (such as time or altitude).
  • The transform from Cartesian to Eucledian coordinates is not needed as we use our own KD-Tree implementation which builds haversine rectangles. Depending on use cases this isn’t always the fastest approach, but it does sidestep some nasty dateline issues.
  • In the case where only the nearest points in the horizontal is needed the collocation falls back the fast single point lookup.
  • The KD-Tree implementation is (relatively well) separated so could easily be switched out for cKDtree or pyresample implementations
  • There are a some tests too, although no docs yet.

I’ll try and put together a notebook building on the above blogpost so that the similarities and differences are a bit clearer.

I’m not familiar enough with xarray indexing to be able to say how well this would fit inside xarray, but hopefully it will be useful before we’re able to crack KD-MultiIndexes!

1reaction
benbovycommented, Jan 5, 2018

Note that it will probably be easier to implement such KDTreeIndex after having refactored indexes and multi-indexes in xarray (see #1603). I think this refactoring would represent a good amount of work, though, so maybe we can do it after if you don’t want to wait too long for the KD-Tree feature?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Pointwise Plugin SDK: runtimeWrite.cxx Source File - Pointwise, Inc.
52 // // index of the vertex as written to the exported mesh file. Currently,. 53 // // only indices that are part...
Read more >
Configuring indexes with the REST API - Google Cloud
This page describes how to create, delete, and list Datastore mode composite indexes using the Datastore mode REST API.
Read more >
Indexing and selecting data — xarray 0.7.1 documentation
Indexing a DataArray directly works (mostly) just like it does for numpy arrays, ... See Pointwise indexing for how to achieve this functionality...
Read more >
dask.array.Array.vindex - Dask documentation
This is equivalent to numpy's advanced indexing, using arrays that are broadcast against each other. This allows for pointwise indexing:.
Read more >
Assigning particular elements of DataArray based on another
You're very close! Instead of boolean indexing, you can use xarray.where() with three arguments: >>> xr.where(ds.coords['q-index'] == 0, ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found