Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Test failures for `gges` and `qz` for float32 input in macOS CI

See original GitHub issue

These started showing up consistently over the last day (example log). Only in the macOS tests (meson) job. That job uses conda-forge, the root cause is very likely the new build (0.23.1 build _3) of OpenBLAS pushed to https://anaconda.org/conda-forge/openblas about 16 hours ago.

___________________________ test_gges_tgexc[float32] ___________________________
[gw0] darwin -- Python 3.10.6 /Users/runner/miniconda3/envs/scipy-dev/bin/python
scipy/linalg/tests/test_lapack.py:3087: in test_gges_tgexc
    assert_allclose(q @ s @ z.conj().T, a, rtol=0, atol=atol)
E   AssertionError: 
E   Not equal to tolerance rtol=0, atol=1.19209e-05
E   
E   Mismatched elements: 100 / 100 (100%)
E   Max absolute difference: 0.5596407
E   Max relative difference: 45.19812
E    x: array([[ 0.269565,  0.510163,  0.307816,  0.720287,  0.713599,  0.25576 ,
E            0.131274,  0.843207,  0.835774,  1.005878],
E          [ 0.364995, -0.002645,  0.148027,  0.409446,  0.200136,  0.596843,...
E    y: array([[0.191519, 0.622109, 0.437728, 0.785359, 0.779976, 0.272593,
E           0.276464, 0.801872, 0.958139, 0.875933],
E          [0.357817, 0.500995, 0.683463, 0.712702, 0.370251, 0.561196,...
        a          = array([[0.19151945, 0.62210876, 0.43772775, 0.7853586 , 0.77997583,
        0.2725926 , 0.27646425, 0.8018722 , 0.9581...43 , 0.9514288 , 0.48035917,
        0.50255954, 0.53687817, 0.81920207, 0.05711564, 0.66942173]],
      dtype=float32)
        atol       = 1.1920928955078125e-05
        b          = array([[0.7671166 , 0.70811534, 0.7968672 , 0.55776083, 0.9658365 ,
        0.1471569 , 0.029647  , 0.59389347, 0.1140...526, 0.7370865 , 0.12702939,
        0.3696499 , 0.604334  , 0.10310444, 0.8023742 , 0.94555324]],
      dtype=float32)
        d1         = -10.973611
        d2         = 0.65421426
        dtype      = <class 'numpy.float32'>
        gges       = <fortran object>
        n          = 10
        q          = array([[ 0.36440596,  0.14539199, -0.48573047,  0.18267933, -0.10118745,
         0.15227516, -0.48237514,  0.06109872....19003534, -0.15249437,
        -0.1685543 , -0.12377075, -0.35968414, -0.3202702 , -0.10042791]],
      dtype=float32)
        result     = (array([[-1.00654995e+00, -1.57352880e-01, -7.44751692e-02,
        -1.94980763e-03,  1.06006734e-01,  7.20489264e-01,... 0.36549678, 0.9152956 ,
       1.0120404 , 0.9464567 , 0.94311225, 0.        , 0.9372638 ],
      dtype=float32), ...)
        s          = array([[-1.00654995e+00, -1.57352880e-01, -7.44751692e-02,
        -1.94980763e-03,  1.06006734e-01,  7.20489264e-01,
...,  0.00000000e+00,
         0.00000000e+00,  0.00000000e+00,  0.00000000e+00,
         3.35054606e-01]], dtype=float32)
        t          = array([[ 0.09172459,  0.18612614, -0.10655626, -0.10534273, -0.5556871 ,
         0.8352932 , -0.11271314,  0.13819972....        ,  0.        ,
         0.        ,  0.        ,  0.        ,  0.        ,  0.9372638 ]],
      dtype=float32)
        tgexc      = <fortran object>
        z          = array([[ 0.45143077,  0.07711446,  0.04053342, -0.06187871, -0.2751032 ,
        -0.56598544, -0.22434531, -0.289968  ....21228388, -0.15313199,
         0.13994187, -0.6340214 ,  0.16412702,  0.56076413,  0.37706485]],
      dtype=float32)
___________________________ test_gges_tgsen[float32] ___________________________
[gw0] darwin -- Python 3.10.6 /Users/runner/miniconda3/envs/scipy-dev/bin/python
scipy/linalg/tests/test_lapack.py:3257: in test_gges_tgsen
    assert_allclose(q @ s @ z.conj().T, a, rtol=0, atol=atol)
E   AssertionError: 
E   Not equal to tolerance rtol=0, atol=1.19209e-05
E   
E   Mismatched elements: 100 / 100 (100%)
E   Max absolute difference: 0.5596407
E   Max relative difference: 45.19812
E    x: array([[ 0.269565,  0.510163,  0.307816,  0.720287,  0.713599,  0.25576 ,
E            0.131274,  0.843207,  0.835774,  1.005878],
E          [ 0.364995, -0.002645,  0.148027,  0.409446,  0.200136,  0.596843,...
E    y: array([[0.191519, 0.622109, 0.437728, 0.785359, 0.779976, 0.272593,
E           0.276464, 0.801872, 0.958139, 0.875933],
E          [0.357817, 0.500995, 0.683463, 0.712702, 0.370251, 0.561196,...
        a          = array([[0.19151945, 0.62210876, 0.43772775, 0.7853586 , 0.77997583,
        0.2725926 , 0.27646425, 0.8018722 , 0.9581...43 , 0.9514288 , 0.48035917,
        0.50255954, 0.53687817, 0.81920207, 0.05711564, 0.66942173]],
      dtype=float32)
        atol       = 1.1920928955078125e-05
        b          = array([[0.7671166 , 0.70811534, 0.7968672 , 0.55776083, 0.9658365 ,
        0.1471569 , 0.029647  , 0.59389347, 0.1140...526, 0.7370865 , 0.12702939,
        0.3696499 , 0.604334  , 0.10310444, 0.8023742 , 0.94555324]],
      dtype=float32)
        d1         = -10.973611
        d2         = 0.65421426
        dtype      = <class 'numpy.float32'>
        gges       = <fortran object>
        n          = 10
        q          = array([[ 0.36440596,  0.14539199, -0.48573047,  0.18267933, -0.10118745,
         0.15227516, -0.48237514,  0.06109872....19003534, -0.15249437,
        -0.1685543 , -0.12377075, -0.35968414, -0.3202702 , -0.10042791]],
      dtype=float32)
        result     = (array([[-1.00654995e+00, -1.57352880e-01, -7.44751692e-02,
        -1.94980763e-03,  1.06006734e-01,  7.20489264e-01,... 0.36549678, 0.9152956 ,
       1.0120404 , 0.9464567 , 0.94311225, 0.        , 0.9372638 ],
      dtype=float32), ...)
        s          = array([[-1.00654995e+00, -1.57352880e-01, -7.44751692e-02,
        -1.94980763e-03,  1.06006734e-01,  7.20489264e-01,
...,  0.00000000e+00,
         0.00000000e+00,  0.00000000e+00,  0.00000000e+00,
         3.35054606e-01]], dtype=float32)
        t          = array([[ 0.09172459,  0.18612614, -0.10655626, -0.10534273, -0.5556871 ,
         0.8352932 , -0.11271314,  0.13819972....        ,  0.        ,
         0.        ,  0.        ,  0.        ,  0.        ,  0.9372638 ]],
      dtype=float32)
        tgsen      = <fortran object>
        tgsen_lwork = <fortran object>
        z          = array([[ 0.45143077,  0.07711446,  0.04053342, -0.06187871, -0.2751032 ,
        -0.56598544, -0.22434531, -0.289968  ....21228388, -0.15313199,
         0.13994187, -0.6340214 ,  0.16412702,  0.56076413,  0.37706485]],
      dtype=float32)
____________________________ TestQZ.test_qz_single _____________________________
[gw1] darwin -- Python 3.10.6 /Users/runner/miniconda3/envs/scipy-dev/bin/python
scipy/linalg/tests/test_decomp.py:2039: in test_qz_single
    assert_array_almost_equal(Q @ AA @ Z.T, A, decimal=5)
E   AssertionError: 
E   Arrays are not almost equal to 5 decimals
E   
E   Mismatched elements: 25 / 25 (100%)
E   Max absolute difference: 1.9282217
E   Max relative difference: 12.652091
E    x: array([[ 0.25942,  0.42247,  0.29118,  1.3622 ,  0.07058],
E          [ 1.54704,  1.04254,  0.65235, -1.17932,  1.4969 ],
E          [ 0.37006,  0.76962, -0.09774,  1.07189, -0.13269],...
E    y: array([[0.92962, 0.31638, 0.18392, 0.20456, 0.56773],
E          [0.59554, 0.96451, 0.65318, 0.74891, 0.65357],
E          [0.74771, 0.96131, 0.00839, 0.10644, 0.2987 ],...
        A          = array([[0.9296161 , 0.31637555, 0.18391882, 0.20456028, 0.567725  ],
       [0.5955447 , 0.9645145 , 0.6531771 , 0.748...93, 0.9646476 , 0.7236853 ],
       [0.6424753 , 0.7174536 , 0.467599  , 0.32558468, 0.4396446 ]],
      dtype=float32)
        AA         = array([[ 0.39677864, -0.87905234,  0.31681818,  0.25420108, -0.31528485],
       [ 0.        ,  2.6049669 ,  0.5027338...10585941, -0.06299087],
       [ 0.        ,  0.        ,  0.        ,  0.        , -0.35764655]],
      dtype=float32)
        B          = array([[0.72968906, 0.99401456, 0.6768737 , 0.7908225 , 0.17091426],
       [0.02684928, 0.8003702 , 0.9037225 , 0.024...4 , 0.09596852, 0.21895005],
       [0.25871906, 0.46810576, 0.4593732 , 0.7095098 , 0.178053  ]],
      dtype=float32)
        BB         = array([[ 0.0964531 , -0.7481051 ,  0.0091472 , -0.6136254 ,  0.07135387],
       [ 0.        ,  2.5299823 ,  0.       ...        ,  0.01339715],
       [ 0.        ,  0.        ,  0.        ,  0.        ,  0.80180556]],
      dtype=float32)
        Q          = array([[-0.5236176 ,  0.43169242,  0.54998064, -0.28841138, -0.39218202],
       [ 0.10638957,  0.3615428 , -0.7044734...05049007,  0.65567476],
       [-0.04425064,  0.36560142, -0.14396492,  0.8444302 , -0.36137164]],
      dtype=float32)
        Z          = array([[-4.1415977e-01,  4.8910376e-01,  6.0941523e-01,  2.2133094e-01,
         4.1094345e-01],
       [ 8.1124866e-0...e-01],
       [-2.1641581e-01,  2.0534211e-01, -2.5105488e-01, -8.4537733e-01,
         3.6511230e-01]], dtype=float32)
        n          = 5
        self       = <scipy.linalg.tests.test_decomp.TestQZ object at 0x12de02ad0>
=========================== short test summary info ============================
FAILED scipy/linalg/tests/test_lapack.py::test_gges_tgexc[float32] - Assertio...
FAILED scipy/linalg/tests/test_lapack.py::test_gges_tgsen[float32] - Assertio...
FAILED scipy/linalg/tests/test_decomp.py::TestQZ::test_qz_single - AssertionE...

Issue Analytics

State:
Created a year ago
Comments:15 (9 by maintainers)

Top GitHub Comments

2reactions

rgommerscommented, Sep 2, 2022

Note that qz is implemented in Python and uses gges internally, so there is only one problem here it looks like gges with float32 input.

1reaction

rgommerscommented, Sep 20, 2022

The issue in CI was fixed by xfail-ing the relevant tests in gh-17057. The sgges issue needs reassessing before the next release. Things seem to work with OpenBLAS 0.3.18, but are broken with 0.3.20/21. As long as we continue to ship 0.3.18 in our own wheels we should be good there - but that doesn’t help conda-forge, Debian et al.