Revise version combinations in CI

[Edit 2021-11-01]: While the particulars below are not necessarily relevant, but the same issue stands for newer versions, e.g py 3.9 and 3.10 at the time of this update, and maybe also for numpy/astropy versions.

Currently I see several errors when running the tests with new numpy (dev or 1.17.0rc1) and new astropy (4.0dev). Looking around in the CI configs it seems that neither there are not tests with numpy dev, as well as there is abundant python 3.5 testing but less for python 3.7 (e.g astropy dev will switch to be python 3.6+ soon, so no point testing it with python3.5).

Also, I would suggest to revise the matrix with the aim to reduce a bit the footprint. E.g. at least use stages, e.g. if one generic build fails then don’t run the whole matrix as the PR needs further commits anyway.

=========================================================================== FAILURES ===========================================================================
_____________________________________________________________________ test_energy_logspace _____________________________________________________________________

i = 2, stacklevel = 3

    def _index_deprecate(i, stacklevel=2):
        try:
>           i = operator.index(i)
E           TypeError: 'numpy.float64' object cannot be interpreted as an integer

/usr/local/lib/python3.7/site-packages/numpy/core/function_base.py:22: TypeError

During handling of the above exception, another exception occurred:

    def test_energy_logspace():
        energy = energy_logspace(emin="0.1 TeV", emax="10 TeV", nbins=3)
        assert energy.unit == "TeV"
        assert_allclose(energy.value, [0.1, 1, 10])
    
        energy = energy_logspace(emin=0.1, emax=10, nbins=3, unit="TeV")
        assert energy.unit == "TeV"
        assert_allclose(energy.value, [0.1, 1, 10])
    
>       energy = energy_logspace(emin="0.1 TeV", emax="10 TeV", nbins=1, per_decade=True)

gammapy/utils/tests/test_energy.py:16: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
gammapy/utils/energy.py:36: in energy_logspace
    energy = np.logspace(x_min, x_max, nbins)
<__array_function__ internals>:6: in logspace
    ???
/usr/local/lib/python3.7/site-packages/numpy/core/function_base.py:277: in logspace
    y = linspace(start, stop, num=num, endpoint=endpoint, axis=axis)
<__array_function__ internals>:6: in linspace
    ???
/usr/local/lib/python3.7/site-packages/numpy/core/function_base.py:128: in linspace
    num = _index_deprecate(num)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

i = 2, stacklevel = 3

    def _index_deprecate(i, stacklevel=2):
        try:
            i = operator.index(i)
        except TypeError:
            msg = ("object of type {} cannot be safely interpreted as "
                   "an integer.".format(type(i)))
            i = int(i)
            stacklevel += 1
>           warnings.warn(msg, DeprecationWarning, stacklevel=stacklevel)
E           DeprecationWarning: object of type <class 'numpy.float64'> cannot be safely interpreted as an integer.

/usr/local/lib/python3.7/site-packages/numpy/core/function_base.py:28: DeprecationWarning
====================================================== 1 failed, 858 passed, 463 skipped in 39.26 seconds ======================================================

[cdeil]

@bsipocz - We should set up better CI and CD for Gammapy. Yes, we’ll want to have extended CI that doesn’t run all the time, but does packaging and science validation and benchmarking and will take hours. PRs are welcome any time of course, but at least I don’t have time for this now, but plan to work on it in the coming months.

[bsipocz]

I would also use pip to add a new “informative/allowed failures” build in the CI to test dev versions of dependencies, which ones? the core deps?

yes. at minimum numpy and astropy, but I would bring in any dependencies that ever caused you regressions or issues with a new release. And if you don’t yet do it, I would raise all deprecation warnings (in fact all warnings) as exceptions, that would force you to fix them as soon as they cause problems.