Parallel regridding support

“Parallel regridding” could mean two things: (see #2 for more about this two-step regridding)

1. Generate regridding weights in parallel.
2. Apply regridding weights in parallel.

The native parallel support in ESMF/ESMPy is based on MPI and horizontal domain decomposition. It works for both generating and applying weights. See https://github.com/nawendt/esmpy-tutorial/blob/master/esmpy_mpi_example.py as an example.

MPI-based horizontal domain decomposition makes perfect sense for earth system model simulation, but for data analysis I would absolutely want to avoid MPI’s complexity. With dask, there’s a simple way to go:

1. Avoid horizontal domain decomposition and only parallelize over additional dimensions like time and level. Writing such code withdask.array will be trivial.
2. Still generate regridding weights in serial. My impression is people tend to have a fixed pair of source and destination grids and regrid a lot of data fields between them. In this case, weights generation only needs to be done once. So we would only care about applying the weights in parallel.

Is there any case that we have to parallelize over horizontal dimensions?

PS: Need to profile the regridding on very large data sets and figure out the bottleneck (generating vs applying weights) before starting to implement anything.

[mrocklin]

The current version (v0.1.1) only runs in serial, although extra-dimension-parallelism should be quite easy to add (literally just a parallel sparse .dot()) May add an experimental parallel method in the next release.

I think that XArray will handle things for you if you use methods like apply_ufunc

Thanks for the suggestion. I haven’t needed to regrid any data that is large enough and requires a distributed cluster. If there’s any specific research needs I am willing to try.

We’re certainly dealing with multi-terabyte datasets for which distributed computation isn’t necessary, but is certainly convenient. I’ll be giving a couple of talks about distributed XArray for data analysis in the next couple of weeks. It would be nice to point to this work as an example of an active community developing functionality with serial computation in mind that “just works” without having to think much about distributed computation. I believe that XArray’s apply_ufunc was designed with this kind of thing in mind.

[JiaweiZhuang]

v0.2 now supports parallel regridding with dask.

Distributed regridding is left to pangeo-data/pangeo#334