question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

io.fits and Lomb-Scargle test failures on ppc64le

See original GitHub issue

I noticed the following test failures on ppc64le:

__________________ TestChecksumFunctions.test_groups_hdu_data __________________

self = <astropy.io.fits.tests.test_checksum.TestChecksumFunctions object at 0x4111f63668>

    def test_groups_hdu_data(self):
        imdata = np.arange(100.0)
        imdata.shape = (10, 1, 1, 2, 5)
        pdata1 = np.arange(10) + 0.1
        pdata2 = 42
        x = fits.hdu.groups.GroupData(imdata, parnames=['abc', 'xyz'],
                                      pardata=[pdata1, pdata2], bitpix=-32)
        hdu = fits.GroupsHDU(x)
        hdu.writeto(self.temp('tmp.fits'), overwrite=True, checksum=True)
        with fits.open(self.temp('tmp.fits'), checksum=True) as hdul:
            assert comparerecords(hdul[0].data, hdu.data)
            assert 'CHECKSUM' in hdul[0].header
>           assert hdul[0].header['CHECKSUM'] == '3eDQAZDO4dDOAZDO'
E           AssertionError: assert 'AAGND29NA8GNA89N' == '3eDQAZDO4dDOAZDO'
E             - 3eDQAZDO4dDOAZDO
E             + AAGND29NA8GNA89N

../../.tox/py37-test/lib/python3.7/site-packages/astropy/io/fits/tests/test_checksum.py:145: AssertionError
______________________ TestDiff.test_different_table_data ______________________

self = <astropy.io.fits.tests.test_diff.TestDiff object at 0x41129ea048>

    def test_different_table_data(self):
        """
        Test diffing table data on columns of several different data formats
        and dimensions.
        """
    
        ca1 = Column('A', format='L', array=[True, False])
        ca2 = Column('B', format='X', array=[[0], [1]])
        ca3 = Column('C', format='4I', dim='(2, 2)',
                     array=[[0, 1, 2, 3], [4, 5, 6, 7]])
        ca4 = Column('D', format='J', bscale=2.0, array=[0.0, 2.0])
        ca5 = Column('E', format='A3', array=['abc', 'def'])
        ca6 = Column('F', format='E', unit='m', array=[0.0, 1.0])
        ca7 = Column('G', format='D', bzero=-0.1, array=[0.0, 1.0])
        ca8 = Column('H', format='C', array=[0.0+1.0j, 2.0+3.0j])
        ca9 = Column('I', format='M', array=[4.0+5.0j, 6.0+7.0j])
        ca10 = Column('J', format='PI(2)', array=[[0, 1], [2, 3]])
    
        cb1 = Column('A', format='L', array=[False, False])
        cb2 = Column('B', format='X', array=[[0], [0]])
        cb3 = Column('C', format='4I', dim='(2, 2)',
                     array=[[0, 1, 2, 3], [5, 6, 7, 8]])
        cb4 = Column('D', format='J', bscale=2.0, array=[2.0, 2.0])
        cb5 = Column('E', format='A3', array=['abc', 'ghi'])
        cb6 = Column('F', format='E', unit='m', array=[1.0, 2.0])
        cb7 = Column('G', format='D', bzero=-0.1, array=[2.0, 3.0])
        cb8 = Column('H', format='C', array=[1.0+1.0j, 2.0+3.0j])
        cb9 = Column('I', format='M', array=[5.0+5.0j, 6.0+7.0j])
        cb10 = Column('J', format='PI(2)', array=[[1, 2], [3, 4]])
    
        ta = BinTableHDU.from_columns([ca1, ca2, ca3, ca4, ca5, ca6, ca7,
                                       ca8, ca9, ca10])
        tb = BinTableHDU.from_columns([cb1, cb2, cb3, cb4, cb5, cb6, cb7,
                                       cb8, cb9, cb10])
    
        diff = TableDataDiff(ta.data, tb.data, numdiffs=20)
        assert not diff.identical
        # The column definitions are the same, but not the column values
        assert diff.diff_columns == ()
        assert diff.diff_values[0] == (('A', 0), (True, False))
        assert diff.diff_values[1] == (('B', 1), ([1], [0]))
        assert diff.diff_values[2][0] == ('C', 1)
        assert (diff.diff_values[2][1][0] == [[4, 5], [6, 7]]).all()
        assert (diff.diff_values[2][1][1] == [[5, 6], [7, 8]]).all()
        assert diff.diff_values[3] == (('D', 0), (0, 2.0))
        assert diff.diff_values[4] == (('E', 1), ('def', 'ghi'))
        assert diff.diff_values[5] == (('F', 0), (0.0, 1.0))
        assert diff.diff_values[6] == (('F', 1), (1.0, 2.0))
        assert diff.diff_values[7] == (('G', 0), (0.0, 2.0))
        assert diff.diff_values[8] == (('G', 1), (1.0, 3.0))
>       assert diff.diff_values[9] == (('H', 0), (0.0+1.0j, 1.0+1.0j))
E       AssertionError: assert (('I', 0), ((4+5j), (5+5j))) == (('H', 0), (1j, (1+1j)))
E         At index 0 diff: ('I', 0) != ('H', 0)
E         Use -v to get the full diff

../../.tox/py37-test/lib/python3.7/site-packages/astropy/io/fits/tests/test_diff.py:556: AssertionError
__________________ TestGroupsFunctions.test_create_groupdata ___________________

self = <astropy.io.fits.tests.test_groups.TestGroupsFunctions object at 0x4111dcb9b0>

    def test_create_groupdata(self):
        """
        Basic test for creating GroupData from scratch.
        """
    
        imdata = np.arange(100.0)
        imdata.shape = (10, 1, 1, 2, 5)
        pdata1 = np.arange(10, dtype=np.float32) + 0.1
        pdata2 = 42.0
        x = fits.hdu.groups.GroupData(imdata, parnames=['abc', 'xyz'],
                                      pardata=[pdata1, pdata2], bitpix=-32)
        assert x.parnames == ['abc', 'xyz']
        assert (x.par('abc') == pdata1).all()
        assert (x.par('xyz') == ([pdata2] * len(x))).all()
>       assert (x.data == imdata).all()
E       assert False
E        +  where False = <built-in method all of numpy.ndarray object at 0x410ff49f30>()
E        +    where <built-in method all of numpy.ndarray object at 0x410ff49f30> = array([[[[[ 0...dtype=float32) == array([[[[[ 0...98., 99.]]]]])
E             Use -v to get the full diff.all

../../.tox/py37-test/lib/python3.7/site-packages/astropy/io/fits/tests/test_groups.py:151: AssertionError
______________ TestImageFunctions.test_scale_bzero_with_int_data _______________

self = <astropy.io.fits.tests.test_image.TestImageFunctions object at 0x4111f51a90>

    def test_scale_bzero_with_int_data(self):
        """
        Regression test for https://github.com/astropy/astropy/issues/4600
        """
    
        a = np.arange(100, 200, dtype=np.int16)
    
        hdu1 = fits.PrimaryHDU(data=a.copy())
        hdu2 = fits.PrimaryHDU(data=a.copy())
        # Previously the following line would throw a TypeError,
        # now it should be identical to the integer bzero case
        hdu1.scale('int16', bzero=99.0)
        hdu2.scale('int16', bzero=99)
>       assert np.allclose(hdu1.data, hdu2.data)
E       assert False
E        +  where False = <function allclose at 0x400327b510>(array([  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n         0,   0,   0,   0,   0,   0,   0,   0,... 0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n         0,   0,   0,   0,   0,  97,  98,  99, 100], dtype=int16), array([  1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,  13,\n        14,  15,  16,  17,  18,  19,  20,  21,...81,  82,  83,  84,  85,  86,  87,  88,  89,  90,  91,\n        92,  93,  94,  95,  96,  97,  98,  99, 100], dtype=int16))
E        +    where <function allclose at 0x400327b510> = np.allclose
E        +    and   array([  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n         0,   0,   0,   0,   0,   0,   0,   0,... 0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n         0,   0,   0,   0,   0,  97,  98,  99, 100], dtype=int16) = <astropy.io.fits.hdu.image.PrimaryHDU object at 0x4111f516a0>.data
E        +    and   array([  1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,  13,\n        14,  15,  16,  17,  18,  19,  20,  21,...81,  82,  83,  84,  85,  86,  87,  88,  89,  90,  91,\n        92,  93,  94,  95,  96,  97,  98,  99, 100], dtype=int16) = <astropy.io.fits.hdu.image.PrimaryHDU object at 0x4111f51780>.data

../../.tox/py37-test/lib/python3.7/site-packages/astropy/io/fits/tests/test_image.py:1059: AssertionError
____________ TestCompressedImage.test_compression_with_gzip_column _____________

self = <astropy.io.fits.tests.test_image.TestCompressedImage object at 0x401b64cba8>

    def test_compression_with_gzip_column(self):
        """
        Regression test for https://github.com/spacetelescope/PyFITS/issues/71
        """
    
        arr = np.zeros((2, 7000), dtype='float32')
    
        # The first row (which will be the first compressed tile) has a very
        # wide range of values that will be difficult to quantize, and should
        # result in use of a GZIP_COMPRESSED_DATA column
        arr[0] = np.linspace(0, 1, 7000)
        arr[1] = np.random.normal(size=7000)
    
        hdu = fits.CompImageHDU(data=arr)
        hdu.writeto(self.temp('test.fits'))
    
        with fits.open(self.temp('test.fits')) as hdul:
            comp_hdu = hdul[1]
    
            # GZIP-compressed tile should compare exactly
>           assert np.all(comp_hdu.data[0] == arr[0])
E           assert False
E            +  where False = <function all at 0x4003260378>(array([ 0.000...dtype=float32) == array([ 0.000...dtype=float32)
E            +    where <function all at 0x4003260378> = np.all
E               Use -v to get the full diff)

../../.tox/py37-test/lib/python3.7/site-packages/astropy/io/fits/tests/test_image.py:1659: AssertionError
________ TestCompressedImage.test_scale_bzero_with_compressed_int_data _________

self = <astropy.io.fits.tests.test_image.TestCompressedImage object at 0x41129da2e8>

    def test_scale_bzero_with_compressed_int_data(self):
        """
        Regression test for https://github.com/astropy/astropy/issues/4600
        and https://github.com/astropy/astropy/issues/4588
    
        Identical to test_scale_bzero_with_int_data() but uses a compressed
        image.
        """
    
        a = np.arange(100, 200, dtype=np.int16)
    
        hdu1 = fits.CompImageHDU(data=a.copy())
        hdu2 = fits.CompImageHDU(data=a.copy())
        # Previously the following line would throw a TypeError,
        # now it should be identical to the integer bzero case
        hdu1.scale('int16', bzero=99.0)
        hdu2.scale('int16', bzero=99)
>       assert np.allclose(hdu1.data, hdu2.data)
E       assert False
E        +  where False = <function allclose at 0x400327b510>(array([  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n         0,   0,   0,   0,   0,   0,   0,   0,... 0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n         0,   0,   0,   0,   0,  97,  98,  99, 100], dtype=int16), array([  1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,  13,\n        14,  15,  16,  17,  18,  19,  20,  21,...81,  82,  83,  84,  85,  86,  87,  88,  89,  90,  91,\n        92,  93,  94,  95,  96,  97,  98,  99, 100], dtype=int16))
E        +    where <function allclose at 0x400327b510> = np.allclose
E        +    and   array([  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n         0,   0,   0,   0,   0,   0,   0,   0,... 0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n         0,   0,   0,   0,   0,  97,  98,  99, 100], dtype=int16) = <astropy.io.fits.hdu.compressed.CompImageHDU object at 0x41129da5c0>.data
E        +    and   array([  1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,  13,\n        14,  15,  16,  17,  18,  19,  20,  21,...81,  82,  83,  84,  85,  86,  87,  88,  89,  90,  91,\n        92,  93,  94,  95,  96,  97,  98,  99, 100], dtype=int16) = <astropy.io.fits.hdu.compressed.CompImageHDU object at 0x4113000400>.data

../../.tox/py37-test/lib/python3.7/site-packages/astropy/io/fits/tests/test_image.py:1704: AssertionError

@olebole - have you seen these before in Debian testing?

Issue Analytics

  • State:open
  • Created 3 years ago
  • Comments:6 (6 by maintainers)

github_iconTop GitHub Comments

2reactions
astrofrogcommented, Nov 11, 2020

We only have arm and s390x not PPC. But I can add all three to the GitHub Actions config 😺

1reaction
saimncommented, Feb 22, 2021

Wow, this one is quite surprising. It has nothing to do with compression, cfitsio etc., just some basic arithmetic. https://github.com/astropy/astropy/runs/1949615134?check_suite_focus=true

    def test_scale_bzero_with_int_data(self):
        """
        Regression test for https://github.com/astropy/astropy/issues/4600
        """
    
        a = np.arange(100, 200, dtype=np.int16)
    
        hdu1 = fits.PrimaryHDU(data=a.copy())
        hdu2 = fits.PrimaryHDU(data=a.copy())
        # Previously the following line would throw a TypeError,
        # now it should be identical to the integer bzero case
        hdu1.scale('int16', bzero=99.0)
        hdu2.scale('int16', bzero=99)
>       assert np.allclose(hdu1.data, hdu2.data)
E       assert False
E        +  where False = <function allclose at 0x40039b3e18>(array([  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n         0,   0,   0,   0,   0,   0,   0,   0,... 0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n         0,   0,   0,   0,   0,  97,  98,  99, 100], dtype=int16), array([  1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,  13,\n        14,  15,  16,  17,  18,  19,  20,  21,...81,  82,  83,  84,  85,  86,  87,  88,  89,  90,  91,\n        92,  93,  94,  95,  96,  97,  98,  99, 100], dtype=int16))
E        +    where <function allclose at 0x40039b3e18> = np.allclose
E        +    and   array([  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n         0,   0,   0,   0,   0,   0,   0,   0,... 0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n         0,   0,   0,   0,   0,  97,  98,  99, 100], dtype=int16) = <astropy.io.fits.hdu.image.PrimaryHDU object at 0x412d4e7198>.data
E        +    and   array([  1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,  13,\n        14,  15,  16,  17,  18,  19,  20,  21,...81,  82,  83,  84,  85,  86,  87,  88,  89,  90,  91,\n        92,  93,  94,  95,  96,  97,  98,  99, 100], dtype=int16) = <astropy.io.fits.hdu.image.PrimaryHDU object at 0x412d4e70f0>.data
Read more comments on GitHub >

github_iconTop Results From Across the Web

Fast Lomb-Scargle Periodograms in Python
Let's use scipy's algorithm to plot the periodogram of the data shown above. Note that the results will not be identical, because this...
Read more >
python-astropy-4.0.2-bp153.1.10 - SUSE Package Hub
[#9929] * astropy.io.fits Fix integer formats of TFORMn=Iw columns in ASCII ... actually detect unit test failures with sys.exit(astropy.tests()) - do not ...
Read more >
Lomb-Scargle Periodograms — Astropy v5.2
The Lomb-Scargle periodogram is designed to detect periodic signals in unevenly spaced observations.
Read more >
Mailman 3 January 2019 - SciPy-Dev - python.org
Build failures (PR 9523) by Blair Azzopardi 30 Jan '19. 30 Jan '19 ... ppc64le linux in CI ... FAIL scipy/stats/tests/test_stats.py::TestIQR::test_scale ?
Read more >
scipy.signal.lombscargle — SciPy v1.9.3 Manual
The Lomb-Scargle periodogram was developed by Lomb [1] and further extended by Scargle [2] to find, and test the significance of weak periodic...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found