Usage documentation?

Sorry to spam you all with multiple issues, but is there any usage documentation associated with your s2cnn package? In particular, I’m curious when to use the different grid types and convolution types. Things like so3_equatorial_grid() vs. so3_soft_grid(), etc.

I also notice you explicitly call so3_integrate() in the MNIST example. I am wondering why there is a need for explicit integration, and what the operation is doing (I couldn’t find that in the papers).

[tscohen]

Regarding the different grids used for the kernel: you are free to use either one, or define your own grid. Spherical CNNs are very new, so we don’t yet know what kind of grid / kernel support is appropriate for each kind of task. There is probably a lot that can be improved in terms of architecture details (for 2D CNNs, it took years to find good architectures, and they’re still improving).

You can think of so3_integrate as being analogous to “global average pooling” in a standard CNN. This is sometimes done at the end to get approximate translation invariance (in the spherical CNN so3_integrate leads to rotation invariance). The reason you can’t just sum up/average the values at all the points is that the grid points are not spread uniformly over the sphere, so we have to weigh them by the inverse of the density, which is given by the Haar measure. For instance, in the SOFT grid we have a lot of points near the north pole. Now imagine a signal that has high values near the north pole. After we rotate the signal, the large values could be near the equator, in which case we’d have fewer points with a high value. Simply summing the original and rotated pixels would not be rotation invariant, but so3_integrate() would be.

[mariogeiger]

• so3_equatorial_grid and so3_near_identity_grid are used to define the kernels support
• so3_integrate integrate a signal on SO3 using the Haar measure

so3_soft_grid defines the SOFT grid that we use to represent our signals. You can use it to compute the Fourier transform of a signal, but we optimized the special case of the SOFT grid using FFT. The following two codes does the same thing (but the second one is much faster)

Slow

grid = so3_soft_grid(b_in)
so3_rft(x, b_out, grid)

Fast

so3_rfft(x, b_out=b_out)