Failing TLS test on python 3.7

A previously working TLS + tornado comms test started failing on Python 3.7 only. Copying error here:

____________________ test_tls_comm_closed_explicit[tornado] ____________________

tcp = <module 'distributed.comm.tcp' from '/home/runner/work/distributed/distributed/distributed/comm/tcp.py'>

    @pytest.mark.asyncio
    async def test_tls_comm_closed_explicit(tcp):
>       await check_comm_closed_explicit("tls://127.0.0.1", **tls_kwargs)

distributed/comm/tests/test_comms.py:774: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
distributed/comm/tests/test_comms.py:753: in check_comm_closed_explicit
    await b_read
distributed/comm/tcp.py:205: in read
    frames_nbytes = await stream.read_bytes(fmt_size)
/usr/share/miniconda3/envs/dask-distributed/lib/python3.7/site-packages/tornado/iostream.py:441: in read_bytes
    self._try_inline_read()
/usr/share/miniconda3/envs/dask-distributed/lib/python3.7/site-packages/tornado/iostream.py:913: in _try_inline_read
    pos = self._read_to_buffer_loop()
/usr/share/miniconda3/envs/dask-distributed/lib/python3.7/site-packages/tornado/iostream.py:815: in _read_to_buffer_loop
    if self._read_to_buffer() == 0:
/usr/share/miniconda3/envs/dask-distributed/lib/python3.7/site-packages/tornado/iostream.py:945: in _read_to_buffer
    bytes_read = self.read_from_fd(buf)
/usr/share/miniconda3/envs/dask-distributed/lib/python3.7/site-packages/tornado/iostream.py:1690: in read_from_fd
    return self.socket.recv_into(buf)
/usr/share/miniconda3/envs/dask-distributed/lib/python3.7/ssl.py:1071: in recv_into
    return self.read(nbytes, buffer)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <ssl.SSLSocket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6>
len = 65536
buffer = bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x...0\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')

    def read(self, len=1024, buffer=None):
        """Read up to LEN bytes and return them.
        Return zero-length string on EOF."""
    
        self._checkClosed()
        if self._sslobj is None:
            raise ValueError("Read on closed or unwrapped SSL socket.")
        try:
            if buffer is not None:
>               return self._sslobj.read(len, buffer)
E               ssl.SSLError: [SSL: KRB5_S_TKT_NYV] unexpected eof while reading (_ssl.c:2570)

/usr/share/miniconda3/envs/dask-distributed/lib/python3.7/ssl.py:929: SSLError

Raising this to track the issue.

[jcrist]

Yeah I suspect this is issue ( openssl/openssl#11378 ), which is referenced in the thread James linked. In OpenSSL 1.1.1e, this behavior was introduced, but in 1.1.1f it was reverted. IIUC the behavior was kept in 3.0.0

Sure, that’s likely the issue. But since no code has changed in python or openssl to lead to this (these versions are all old), I’m more interested in what changed in the conda package metadata that led us to pull in a different version of openssl all of a sudden.

Is the issue with Python 3.7 or with Tornado 5?

The asyncio comms properly handles this issue, so it’s likely tornado not properly handling the potential variation in failure modes here.

---

Since we should drop python 3.7 soon-ish (per NEP-29), I’m just going to push up a patch that xfails this test for python 3.7.

[jakirkham]

I’m not sure

xfailing sounds reasonable