v3.1rc1 - test_arithmetic_overload_ccddata_operand failure

(py37) ❯ pytest astropy/nddata -k test_arithmetic_overload_ccddata_operand                                                                                                                     
/home/conseil/miniconda3/envs/py37/lib/python3.7/site-packages/matplotlib/__init__.py:886: MatplotlibDeprecationWarning:                                                                       
examples.directory is deprecated; in the future, examples will be found relative to the 'datapath' directory.                                                                                  
  "found relative to the 'datapath' directory.".format(key))                                                                                                                                   
/home/conseil/miniconda3/envs/py37/lib/python3.7/importlib/_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__
  return f(*args, **kwds)
===================================================================================== test session starts ====================================================================================$
platform linux -- Python 3.7.1, pytest-3.10.1, py-1.7.0, pluggy-0.8.0

Running tests with Astropy version 3.1rc1.
Running tests in astropy/nddata.

Date: 2018-11-16T18:14:05

Platform: Linux-4.16.0-0.bpo.1-amd64-x86_64-with-debian-9.6

Executable: /home/conseil/miniconda3/envs/py37/bin/python

Full Python Version:
3.7.1 (default, Oct 23 2018, 19:19:42)
[GCC 7.3.0]

encodings: sys: utf-8, locale: UTF-8, filesystem: utf-8
byteorder: little
float info: dig: 15, mant_dig: 15

Numpy: 1.15.4
Scipy: 1.1.0
Matplotlib: 3.0.2
h5py: not available
Pandas: 0.23.4
Cython: 0.29
astropy_helpers: 1.1rc1
Using Astropy options: remote_data: none.

rootdir: /home/conseil/lib/astropy, inifile: setup.cfg
plugins: xdist-1.24.1, remotedata-0.3.1, openfiles-0.3.0, forked-0.2, doctestplus-0.2.0, cov-2.6.0, arraydiff-0.2
collected 587 items / 586 deselected                                                                                                                                                          

astropy/nddata/tests/test_ccddata.py F              
========================================================================================== FAILURES ===========================================================================================
__________________________________________________________________________ test_arithmetic_overload_ccddata_operand ___________________________________________________________________________

ccd_data = CCDData([[-1.0856306 ,  0.99734545,  0.2829785 , ..., -1.36347154,
           0.37940061, -0.37917643],
         [ 0.6... -0.82190772],
         [-0.66845311, -2.15776154, -0.88186588, ..., -0.47226641,
           0.58196437,  0.97061286]])

    def test_arithmetic_overload_ccddata_operand(ccd_data):
        ccd_data.uncertainty = StdDevUncertainty(np.ones_like(ccd_data))
        operand = ccd_data.copy()
        result = ccd_data.add(operand)
        assert len(result.meta) == 0
        np.testing.assert_array_equal(result.data,
                                      2 * ccd_data.data)
        np.testing.assert_array_equal(result.uncertainty.array,
>                                     np.sqrt(2) * ccd_data.uncertainty.array)
E       AssertionError: 
E       Arrays are not equal
E                                        
E       (mismatch 100.0%)
E        x: array([1.414214, 1.414214, 1.414214, ..., 1.414214, 1.414214, 1.414214])
E        y: array([1.414214, 1.414214, 1.414214, ..., 1.414214, 1.414214, 1.414214])                             
                                                                                                                                                                                              
astropy/nddata/tests/test_ccddata.py:472: AssertionError
========================================================================== 1 failed, 586 deselected in 1.82 seconds ===========================================================================

[MSeifert04]

Since it doesn’t fail on my computer it seems like it’s probably not significant and almost certainly not a blocker.

Maybe we could check what happens if we use assert_array_almost_equal_nulp instead of assert_array_equal. If that works with nulp=1 (the default) or nulp=2 (should also be okay) we can definitely exclude “significant deviation”.

[MSeifert04]

Yes, interestingly this one doesn’t tip over CI, or my local testing either.

Could well be a non-standard sqrt implementation on the system. As far as I know there are some limits that a IEEE-conforming implementation should satisfy. But I’m really not an expert on this field so I’m not sure if that’s still conforming (meaning our test was too strict) or not (meaning their tests are not strict enough or they intentionally traded that bit of precision for something else).

As far as I’m concerned one-ulp deviation is “good enough” for uncertainty propagation.