Make scipy a required dependency

I would like to suggest making scipy a required dependency of the core package (or some alternative approaches mentioned below).

There are a number of sub-packages in astropy where we use scipy where it’s not obvious that would be the case and where users will encounter unexpected ImportErrors if it is not installed. For instance the following code snippets from the docs require scipy:

astropy.coordinates

>>> c = SkyCoord(ra=ra1*u.degree, dec=dec1*u.degree, distance=distance1*u.kpc)
>>> catalog = SkyCoord(ra=ra2*u.degree, dec=dec2*u.degree, distance=distance2*u.kpc)
>>> idx, d2d, d3d = c.match_to_catalog_3d(catalog)  # REQUIRES SCIPY

astropy.cosmology

>>> from astropy.cosmology import WMAP9 as cosmo
>>> cosmo.H(0)
<Quantity 69.32 km / (Mpc s)>
>>> cosmo.kpc_proper_per_arcmin(3)  # REQUIRES SCIPY
<Quantity 472.97709620405266 kpc / arcmin>

astropy.units

>>> from astropy.cosmology import WMAP9
>>> z = 1100 * cu.redshift
>>> z.to(u.K, cu.with_redshift(WMAP9))
<Quantity 3000.225 K>
>>> z.to(u.Mpc, cu.with_redshift(WMAP9, distance="luminosity"))   # REQUIRES SCIPY
<Quantity 15418438.76317008 Mpc>

astropy.convolution

>>> gauss_oversample = Gaussian1DKernel(3, mode='oversample', factor=10)
>>> gauss_integrate = Gaussian1DKernel(3, mode='integrate')   # REQUIRES SCIPY

astropy.modeling

scipy is also used extensively in modeling!

In all of these examples, there is no way the user would know in advance that scipy is required, and they will stumble on these issues and have to install scipy to get them to work. Now that scipy is needed for some of the functionality in at least five sub-packages, I think it is time to consider requiring it. In total, from scipy. appears 76 times in the astropy core package, and doctest-requires:: scipy appears 36 times!

As packaging has improved tremendously since the start of the project, and wheels and conda packages are available for scipy for many platforms, I think it would be reasonable to start requiring scipy and would result in fewer surprise errors for users.

The philosophy here is the same as why we added PyYAML as a required dependency - in many cases users couldn’t have suspected this dependency.

To give a counter example of something I think doesn’t need to be a required dependency, astropy.visualization provides lots of functionality to expand matplotlib, but in many cases matplotlib needs to be installed anyway for the user to want to use the functionality. Take a look at the following example:

    from astropy import units as u
    from astropy.visualization import quantity_support
    quantity_support()
    from matplotlib import pyplot as plt
    plt.figure(figsize=(5,3))
    plt.plot([1, 2, 3] * u.m)

Here matplotlib is imported explicitly anyway, and a user wouldn’t be trying to use quantity_support without it installed.

Another counter-example would be the table reader/writer for HDF5 - in this case it’s ok to error if h5py is not installed with an explicit error that says ‘h5py is required to read/write HDF5’ files.

Perhaps this requires an APE or a policy document describing some basic rules/tests we can use to determine if a dependency should be required.

Alternatives

I suspect there will be resistance to including scipy as a required dependency given that it may not be easy to install on some more exotic platforms. So here are some more options:

1. The option mentioned above, add scipy to install_requires
2. In a lot of cases we might only be using a very limited subset of scipy, eg integration. We could try and replace scipy with another lighter weight dependency in these cases and require that.
3. We could make scipy a required dependency only on the main common platforms where wheels are available (this can be done in the requirements with environment markers) and make sure that the test suite still passes without scipy installed
4. We could require scipy as in option 1. but document how users can install astropy without it (–no-deps and explicitly specifying our minimal required dependencies) and continue to include some test builds without scipy
5. We could make it so that sub-packages don’t import at all if scipy isn’t installed - for example cosmology uses scipy enough that we could just disable importing the whole subpackage if scipy is not installed (rather than giving surprise errors for half the methods).

[astrofrog]

So coming back to this, I’d like to suggest following options 3 and 4 - namely that we list scipy as a required dependency but only on common platforms where wheels are available which I think would satisfy most regular users of astropy while not forcing people on more exotic platforms to build scipy from source. We then would make sure that the test suite can run without scipy installed and give information about installing with --no-deps in the docs. Finally, we would add a comment in the requirements that we recommend that distribution maintainers, e.g. conda, debian etc include scipy as a required or default dependency).

[hamogu]

@robertdstein Thanks for your feedback. That is very valuable to know. One of the problems in (astropy) development is that many users do not tell us about their expectations and many of our conversations start with “A typical user might …”, which is not a very good basis to make the call what dependency should be required vs. optional.