[Tracker] Implement all `scipy.*` APIs in CuPy

Implement GPU version of scipy.* functions in cupyx.scipy.* namespace.

This is a tracker issue that summarizes the implementation status of each SciPy public module in CuPy. See the comparison table for details.

Legends

• 🥚 Module not yet available in CuPy.
• 🐥 Module available in CuPy but coverage needs improvement.
• :shipit: Module available in CuPy and major features are implemented (may still have a few unimplemented APIs).

List of Modules

• 🥚 Clustering package (scipy.cluster)
  • 🥚 K-means clustering and vector quantization (scipy.cluster.vq) (#5947)
  • 🥚 Hierarchical clustering (scipy.cluster.hierarchy) (#3434)
• 🥚 Constants (scipy.constants)
• :shipit: Discrete Fourier transforms (scipy.fft)
• :shipit: Legacy discrete Fourier transforms (scipy.fftpack) (note: deprecated, no further development planned)
• 🥚 Integration and ODEs (scipy.integrate) (#7019)
• 🥚 Interpolation (scipy.interpolate) (#7186)
• 🥚 Input and output (scipy.io)
  • 🥚 scipy.io.arff
  • 🥚 scipy.io.matlab
  • 🥚 scipy.io.wavfile
• 🐥 Linear algebra (scipy.linalg)
  • 🥚 Low-level BLAS functions (scipy.linalg.blas)
  • 🥚 scipy.linalg.cython_blas (note: unlikely supported in CuPy)
  • 🥚 Low-level LAPACK functions (scipy.linalg.lapack)
  • 🥚 scipy.linalg.cython_lapack
  • 🥚 Interpolative matrix decomposition (scipy.linalg.interpolative)
• :shipit: Multidimensional image processing (scipy.ndimage)
• 🥚 Orthogonal distance regression (scipy.odr)
• 🥚 Optimization and root finding (scipy.optimize) (#6112)
• 🐥 Signal processing (scipy.signal)
  • 🥚 scipy.signal.windows
• :shipit: Sparse matrices (scipy.sparse)
  • :shipit: Sparse linear algebra (scipy.sparse.linalg)
  • 🐥 Compressed sparse graph routines (scipy.sparse.csgraph) (#2431)
• 🐥 Spatial algorithms and data structures (scipy.spatial) (#5946)
  • 🐥 Distance computations (scipy.spatial.distance) (#5946)
  • 🥚 Spatial transformations (scipy.spatial.transform)
• 🐥 Special functions (scipy.special)
• 🐥 Statistical functions (scipy.stats)
  • 🥚 Contingency table functions (scipy.stats.contingency) (note: RAFT could be used)
  • 🥚 scipy.stats.distributions
  • 🥚 Statistical functions for masked arrays (scipy.stats.mstats) (note: masked arrays unsupported in CuPy)
  • 🥚 Quasi-Monte Carlo submodule (scipy.stats.qmc)
  • 🥚 Random Number Generators (scipy.stats.sampling) (note: RAFT could be used)

Note: Miscellaneous routines (scipy.misc) is excluded from the list as it is planned to be removed in the upstream (https://github.com/scipy/scipy/issues/15608).

Starter Task

If you are new to CuPy and wondering where to start, here are some good first issues to try. They are relatively simple and independent.

• APIs under Special functions (scipy.special)
• APIs under Statistical Functions (scipy.stats)
• scipy.sparse.block_diag (#7058)
• scipy.sparse.load_npz
• scipy.sparse.save_npz
• scipy.constants.* (hint: most things except for methods can just be borrowed from SciPy)

Steps to Contribute

1. Fork and star ⭐ the CuPy repository 😉

2. Pick a function you want to work on from any of the modules listed above. You can find the function in the SciPy API Reference to understand what should be implemented. If you want to work on new modules (🥚), first discuss with us on this issue.

3. Implement a function in your branch. If you need help, join Gitter or just ask for help in this issue.

4. Don’t forget to write a test code!

5. Build CuPy and run tests to confirm that the function runs fine: pip install -e . && pytest tests/cupyx_tests/scipy_tests/PATH_TO_YOUR_TEST See the Contribution Guide for details.

6. Submit a pull-request to the master branch.

Note that you will need a GPU environment to develop CuPy.

See also: NumPy API Tracker Issue #6078

[rgommers]

This looks great, thanks @kmaehashi!

Miscellaneous routines (scipy.misc)

Note that we’re quite likely deprecating this module in SciPy 1.9.0 (see https://github.com/scipy/scipy/issues/15608), so I suggest not working on that.

[cjnolet]

@kmaehashi, RAFT contains hierarchical clustering code as well: https://github.com/cupy/cupy/issues/3434.

RAFT also contains functions for computing contingency table and sampling from various distributions. Wondering if those might be useful here as well.