Retrieve regridding weights as numpy arrays

Most regridding schemes are linear, i.e. the output data field is linearly dependent on the input data field. Any linear transform can be viewed as a matrix-vector multiplication y = A*x, where A is a matrix (typically sparse) containing regridding weights, and x, y are input and output data fields flatten to 1D.

In practice, linear regridding schemes are broken into two steps:

1. Calculate regridding weights (i.e. get the matrix A), from the knowledge of source and destination grids. In ESMPy, this is done by regrid = ESMF.Regrid(...).
2. Apply regridding weights to the data field (i.e. perform A*x). In ESMPy, this is done by destfield = regrid(sourcefield, destfield), where regrid was created in the previous step.

However, ESMPy’s regrid object is like a black box. It knows how to perform regridding but there’s no way to explicitly view the regridding weights (the matrix A).

In the Fortran version of ESMF, the function ESMF_RegridWeightGen dumps regridding weights to NetCDF files. The content of the file is shown in “12.8 Regrid Interpolation Weight File Format” section in ESMF documention (link). The matrix A is stored in a sparse matrix form by variables row,col and S in that NetCDF file. But in ESMPy there’s no function equivalent to ESMF_RegridWeightGen.

Being able to view the regridding weights in the Python-level will solve many troubles:

• Dimension matching. To broadcast the regridding operation across additional dimensions (call them N1, N2…), ESMPy requires the input data to have the shape [Nlat, Nlon, N1, N2,…]. Instead of using xarray’s transpose() function to match ESMPy’s expection, we can write the sparse matrix multiplication directly in numpy, taking care of dimension broadcasting.
• Dask intergration. By using numpy instead of the underlying Fortran routine to apply regridding weights, we can use dask.array easily and natively. Otherwise, we need to let each dask worker call the underlying Fortran routine separately to regrid each hyberslab – sounds like a very ugly solution.
• Allow other programs to use ESMF regridding. One use case is NASA-GEOS5’s MAPL software. Calculating regridding weights is very hard and we don’t want to rebuild the wheel, but applying the weights to the data field can be done by ~5 lines of code in most languages.

@bekozi seems to have some Python tools for ESMF_RegridWeightGen. Maybe we could start with that.

[JiaweiZhuang]

@bekozi Thanks for looking into this! I am OK with Python2.7 for now.

[bekozi]

My sincere apologies for the delay on getting you the weight file write code. We have a snapshot tag that includes writing weights to file:

git clone -b ESMF_7_1_0_beta_snapshot_35 https://git.code.sf.net/p/esmf/esmf esmf

The only change to your code should be the addition of a filename argument to the regrid call:

rh = ESMF.Regrid(srcfield, dstfield, filename='/tmp/weights.nc', ...)

Please let me know if you have any questions or find any issues! Though tested, note this is considered “development code”.