Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

5.0rc1 expected warnings intercepted by sunpy logger

See original GitHub issue

Description

With Sunpy 3.0.2 or 3.1.0 installed along with Astropy 5.0rc1 , some tests in cosmology and test_logger are failing on missing expected warnings that are apparently intercepted by Sunpy’s logger mechanism.

Expected behavior

Warnings should get through to the pytest.warns context.

Actual behavior

===================================================================================== FAILURES =====================================================================================
_________________________________________________________________________ test_equivalencies_asdf[with_H0] _________________________________________________________________________

tmpdir = local('/private/var/folders/bb/f_z2vsjd21d_p0jpqxzhhwhh00027w/T/pytest-of-derek/pytest-65/test_equivalencies_asdf_with_H0'), equiv = <function with_H0 at 0x10fe41430>

    @pytest.mark.skipif(not HAS_ASDF, reason="requires ASDF")
    @pytest.mark.parametrize("equiv", [cu.with_H0])
    def test_equivalencies_asdf(tmpdir, equiv):
        from asdf.tests import helpers
    
        tree = {"equiv": equiv()}
        with (
            pytest.warns(AstropyDeprecationWarning, match="`with_H0`")
            if equiv.__name__ == "with_H0"
            else contextlib.nullcontext()
        ):
    
>           helpers.assert_roundtrip_tree(tree, tmpdir)
E           Failed: DID NOT WARN. No warnings of type (<class 'astropy.utils.exceptions.AstropyDeprecationWarning'>,) was emitted. The list of emitted warnings is: [].

/opt/lib/python3.8/site-packages/astropy/cosmology/tests/test_units.py:406: Failed
------------------------------------------------------------------------------- Captured stderr call -------------------------------------------------------------------------------
WARNING: AstropyDeprecationWarning: `with_H0` is deprecated from `astropy.units.equivalencies` since astropy 5.0 and may be removed in a future version. Use `astropy.cosmology.units.with_H0` instead. [astropy.units.equivalencies]
WARNING: AstropyDeprecationWarning: `with_H0` is deprecated from `astropy.units.equivalencies` since astropy 5.0 and may be removed in a future version. Use `astropy.cosmology.units.with_H0` instead. [astropy.units.equivalencies]
WARNING: AstropyDeprecationWarning: `with_H0` is deprecated from `astropy.units.equivalencies` since astropy 5.0 and may be removed in a future version. Use `astropy.cosmology.units.with_H0` instead. [astropy.units.equivalencies]
127.0.0.1 - - [05/Nov/2021 23:16:04] "GET /test.asdf HTTP/1.1" 200 -
WARNING: AstropyDeprecationWarning: `with_H0` is deprecated from `astropy.units.equivalencies` since astropy 5.0 and may be removed in a future version. Use `astropy.cosmology.units.with_H0` instead. [astropy.units.equivalencies]
WARNING: AstropyDeprecationWarning: `with_H0` is deprecated from `astropy.units.equivalencies` since astropy 5.0 and may be removed in a future version. Use `astropy.cosmology.units.with_H0` instead. [astropy.units.equivalencies]
WARNING: AstropyDeprecationWarning: `with_H0` is deprecated from `astropy.units.equivalencies` since astropy 5.0 and may be removed in a future version. Use `astropy.cosmology.units.with_H0` instead. [astropy.units.equivalencies]
-------------------------------------------------------------------------------- Captured log call ---------------------------------------------------------------------------------
WARNING  sunpy:logger.py:211 AstropyDeprecationWarning: `with_H0` is deprecated from `astropy.units.equivalencies` since astropy 5.0 and may be removed in a future version. Use `astropy.cosmology.units.with_H0` instead.
WARNING  sunpy:logger.py:211 AstropyDeprecationWarning: `with_H0` is deprecated from `astropy.units.equivalencies` since astropy 5.0 and may be removed in a future version. Use `astropy.cosmology.units.with_H0` instead.
WARNING  sunpy:logger.py:211 AstropyDeprecationWarning: `with_H0` is deprecated from `astropy.units.equivalencies` since astropy 5.0 and may be removed in a future version. Use `astropy.cosmology.units.with_H0` instead.
WARNING  sunpy:logger.py:211 AstropyDeprecationWarning: `with_H0` is deprecated from `astropy.units.equivalencies` since astropy 5.0 and may be removed in a future version. Use `astropy.cosmology.units.with_H0` instead.
WARNING  sunpy:logger.py:211 AstropyDeprecationWarning: `with_H0` is deprecated from `astropy.units.equivalencies` since astropy 5.0 and may be removed in a future version. Use `astropy.cosmology.units.with_H0` instead.
WARNING  sunpy:logger.py:211 AstropyDeprecationWarning: `with_H0` is deprecated from `astropy.units.equivalencies` since astropy 5.0 and may be removed in a future version. Use `astropy.cosmology.units.with_H0` instead.

and for the logger

______________________________________________________________________________ test_log_to_file[None] ______________________________________________________________________________

tmpdir = local('/private/var/folders/bb/f_z2vsjd21d_p0jpqxzhhwhh00027w/T/pytest-of-derek/pytest-64/test_log_to_file_None_0'), level = 'INFO'

    @pytest.mark.parametrize(('level'), [None, 'DEBUG', 'INFO', 'WARN', 'ERROR'])
    def test_log_to_file(tmpdir, level):
    
        local_path = tmpdir.join('test.log')
        log_file = local_path.open('wb')
        log_path = str(local_path.realpath())
        orig_level = log.level
    
        try:
            if level is not None:
                log.setLevel(level)
    
            with log.log_to_file(log_path):
                log.error("Error message")
                log.warning("Warning message")
                log.info("Information message")
                log.debug("Debug message")
    
            log_file.close()
        finally:
            log.setLevel(orig_level)
    
        log_file = local_path.open('rb')
        log_entries = log_file.readlines()
        log_file.close()
    
        if level is None:
            # The log level *should* be set to whatever it was in the config
            level = conf.log_level
    
        # Check list length
        if level == 'DEBUG':
            assert len(log_entries) == 4
        elif level == 'INFO':
            assert len(log_entries) == 3
        elif level == 'WARN':
            assert len(log_entries) == 2
        elif level == 'ERROR':
            assert len(log_entries) == 1
    
        # Check list content
    
>       assert eval(log_entries[0].strip())[-3:] == (
            'astropy.tests.test_logger', 'ERROR', 'Error message')
E         File "<string>", line 1
E           2021-11-05 23:10:45,177, astropy.tests.test_logger, ERROR, Error message
E                   ^
E       SyntaxError: leading zeros in decimal integer literals are not permitted; use an 0o prefix for octal integers

/opt/lib/python3.10/site-packages/astropy/tests/test_logger.py:410: SyntaxError
------------------------------------------------------------------------------- Captured stdout call -------------------------------------------------------------------------------
INFO: Information message [astropy.tests.test_logger]
------------------------------------------------------------------------------- Captured stderr call -------------------------------------------------------------------------------
ERROR: Error message [astropy.tests.test_logger]
WARNING: Warning message [astropy.tests.test_logger]
-------------------------------------------------------------------------------- Captured log call ---------------------------------------------------------------------------------
ERROR    astropy:test_logger.py:381 Error message
WARNING  astropy:test_logger.py:382 Warning message
INFO     astropy:test_logger.py:383 Information message

with corresponding failures for all levels from DEBUG to ERROR, and

______________________________________________________________________________ test_log_to_file_level ______________________________________________________________________________

tmpdir = local('/private/var/folders/bb/f_z2vsjd21d_p0jpqxzhhwhh00027w/T/pytest-of-derek/pytest-64/test_log_to_file_level0')

    def test_log_to_file_level(tmpdir):
    
        local_path = tmpdir.join('test.log')
        log_file = local_path.open('wb')
        log_path = str(local_path.realpath())
    
        with log.log_to_file(log_path, filter_level='ERROR'):
            log.error("Error message")
            log.warning("Warning message")
    
        log_file.close()
    
        log_file = local_path.open('rb')
        log_entries = log_file.readlines()
        log_file.close()
    
        assert len(log_entries) == 1
>       assert eval(log_entries[0].strip())[-2:] == (
            'ERROR', 'Error message')
E         File "<string>", line 1
E           2021-11-05 23:10:45,282, astropy.tests.test_logger, ERROR, Error message
E                   ^
E       SyntaxError: leading zeros in decimal integer literals are not permitted; use an 0o prefix for octal integers

/opt/lib/python3.10/site-packages/astropy/tests/test_logger.py:443: SyntaxError
------------------------------------------------------------------------------- Captured stderr call -------------------------------------------------------------------------------
ERROR: Error message [astropy.tests.test_logger]
WARNING: Warning message [astropy.tests.test_logger]
-------------------------------------------------------------------------------- Captured log call ---------------------------------------------------------------------------------
ERROR    astropy:test_logger.py:433 Error message
WARNING  astropy:test_logger.py:434 Warning message

and a timeout (I haven’t seen this without sunpy installed either, but also do not get it consistently, so might indeed be a connection error):

_________________________________________________________________________________ test_api_lookup __________________________________________________________________________________

    @pytest.mark.remote_data
    def test_api_lookup():
        try:
>           strurl = misc.find_api_page('astropy.utils.misc', 'dev', False,
                                        timeout=5)

/opt/lib/python3.10/site-packages/astropy/utils/tests/test_misc.py:33: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/opt/lib/python3.10/site-packages/astropy/utils/misc.py:243: in find_api_page
    with get_readable_fileobj(url, encoding='binary', remote_timeout=timeout,
/opt/lib/python3.10/contextlib.py:135: in __enter__
    return next(self.gen)
/opt/lib/python3.10/site-packages/astropy/utils/data.py:267: in get_readable_fileobj
    name_or_obj = download_file(
/opt/lib/python3.10/site-packages/astropy/utils/data.py:1396: in download_file
    raise errors[sources[0]]
/opt/lib/python3.10/site-packages/astropy/utils/data.py:1359: in download_file
    f_name = _download_file_from_source(
/opt/lib/python3.10/site-packages/astropy/utils/data.py:1163: in _download_file_from_source
    with _try_url_open(source_url, timeout=timeout, http_headers=http_headers,
/opt/lib/python3.10/site-packages/astropy/utils/data.py:1100: in _try_url_open
    return urlopener.open(req, timeout=timeout)
/opt/lib/python3.10/urllib/request.py:519: in open
    response = self._open(req, data)
/opt/lib/python3.10/urllib/request.py:536: in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
/opt/lib/python3.10/urllib/request.py:496: in _call_chain
    result = func(*args)
/opt/lib/python3.10/urllib/request.py:1377: in http_open
    return self.do_open(http.client.HTTPConnection, req)
/opt/lib/python3.10/urllib/request.py:1352: in do_open
    r = h.getresponse()
/opt/lib/python3.10/http/client.py:1368: in getresponse
    response.begin()
/opt/lib/python3.10/http/client.py:317: in begin
    version, status, reason = self._read_status()
/opt/lib/python3.10/http/client.py:278: in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <socket.SocketIO object at 0x130643700>, b = <memory at 0x137c06e00>

    def readinto(self, b):
        """Read up to len(b) bytes into the writable buffer *b* and return
        the number of bytes read.  If the socket is non-blocking and no bytes
        are available, None is returned.
    
        If *b* is non-empty, a 0 return value indicates that the connection
        was shutdown at the other end.
        """
        self._checkClosed()
        self._checkReadable()
        if self._timeout_occurred:
            raise OSError("cannot read from timed out object")
        while True:
            try:
>               return self._sock.recv_into(b)
E               TimeoutError: timed out

/opt/lib/python3.10/socket.py:705: TimeoutError

Notably the with_H0 warning test in test_has_expected_equivalencies succeeds. Also the logger tests are passing if just called on their own with package='test' or ‘args=’-k test_logger’`.

Steps to Reproduce

Run astropy.test() [all or for package='cosmology'] with sunpy installed in the same environment on any of Python 3.8.11, 3.9.6, 3.10.0.

System Details

Platform: macOS-10.14.6-x86_64-i386-64bit

Executable: /opt/bin/python3.10

Full Python Version: 3.10.0 (default, Oct 5 2021, 03:51:16) [Clang 11.0.0 (clang-1100.0.33.17)]

encodings: sys: utf-8, locale: UTF-8, filesystem: utf-8 byteorder: little float info: dig: 15, mant_dig: 15

Package versions: Numpy: 1.21.2 Scipy: 1.7.1 Matplotlib: 3.5.0rc1 h5py: 3.5.0 Pandas: 1.3.3 PyERFA: 2.0.0 Cython: 0.29.24 Scikit-image: not available asdf: 2.8.1