Python 3.6 RuntimeError: read() called while another coroutine is already waiting for incoming data

Running in python 3.7 and 3.8 have been going beautifully, but now I’ve decided I wanted to add my API client using httpx to a Python 3.6 project, and I’m getting the above error whenever I try to call httpx. Below is the stack trace of one of the calls:

src/emcee/api.py:119: in get_with_retry
    return self.session.get(path, params=params)
.nox/test-3-6/lib/python3.6/site-packages/httpx/client.py:818: in get
    proxies=proxies,
.nox/test-3-6/lib/python3.6/site-packages/httpx/client.py:729: in request
    proxies=proxies,
.nox/test-3-6/lib/python3.6/site-packages/httpx/client.py:760: in send
    async_response = concurrency_backend.run(coroutine, *args, **kwargs)
.nox/test-3-6/lib/python3.6/site-packages/httpx/concurrency/asyncio.py:241: in run
    return self.loop.run_until_complete(coroutine(*args, **kwargs))
../../../.pyenv/versions/3.6.9/lib/python3.6/asyncio/base_events.py:484: in run_until_complete
    return future.result()
.nox/test-3-6/lib/python3.6/site-packages/httpx/client.py:230: in _get_response
    return await get_response(request)
.nox/test-3-6/lib/python3.6/site-packages/httpx/middleware/redirect.py:31: in __call__
    response = await get_response(request)
.nox/test-3-6/lib/python3.6/site-packages/httpx/client.py:191: in get_response
    request, verify=verify, cert=cert, timeout=timeout
.nox/test-3-6/lib/python3.6/site-packages/httpx/dispatch/connection_pool.py:126: in send
    raise exc
.nox/test-3-6/lib/python3.6/site-packages/httpx/dispatch/connection_pool.py:121: in send
    request, verify=verify, cert=cert, timeout=timeout
.nox/test-3-6/lib/python3.6/site-packages/httpx/dispatch/connection.py:65: in send
    response = await self.h11_connection.send(request, timeout=timeout)
.nox/test-3-6/lib/python3.6/site-packages/httpx/dispatch/http11.py:53: in send
    http_version, status_code, headers = await self._receive_response(timeout)
.nox/test-3-6/lib/python3.6/site-packages/httpx/dispatch/http11.py:133: in _receive_response
    event = await self._receive_event(timeout)
.nox/test-3-6/lib/python3.6/site-packages/httpx/dispatch/http11.py:174: in _receive_event
    self.READ_NUM_BYTES, timeout, flag=self.timeout_flag
.nox/test-3-6/lib/python3.6/site-packages/httpx/concurrency/asyncio.py:79: in read
    data = await asyncio.wait_for(self.stream_reader.read(n), read_timeout)
../../../.pyenv/versions/3.6.9/lib/python3.6/asyncio/tasks.py:339: in wait_for
    return (yield from fut)
../../../.pyenv/versions/3.6.9/lib/python3.6/asyncio/streams.py:634: in read
    yield from self._wait_for_data('read')
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <StreamReader t=<asyncio.sslproto._SSLProtocolTransport object at 0x10b508470>>, func_name = 'read'

    @coroutine
    def _wait_for_data(self, func_name):
        """Wait until feed_data() or feed_eof() is called.

        If stream was paused, automatically resume it.
        """
        # StreamReader uses a future to link the protocol feed_data() method
        # to a read coroutine. Running two read coroutines at the same time
        # would have an unexpected behaviour. It would not possible to know
        # which coroutine would get the next data.
        if self._waiter is not None:
            raise RuntimeError('%s() called while another coroutine is '
>                              'already waiting for incoming data' % func_name)
E           RuntimeError: read() called while another coroutine is already waiting for incoming data

../../../.pyenv/versions/3.6.9/lib/python3.6/asyncio/streams.py:452: RuntimeError

Python: 3.6.9 httpx: 0.7.3

It appears to be with the way I’ve instantiated TimeoutConfig (with no values, to allow infinite timeout). Here’s a minimally reproducible example:

>>> import httpx

>>> no_timeout = httpx.config.TimeoutConfig()
>>> session = httpx.Client(timeout=no_timeout)
>>> session.get("https://httpbin.org")
<Response [200 OK]>

Works fine in 3.7+, fails in 3.6

[florimondmanca]

@iluxonchik Can we continue discussion on this in #527?

Also, I think you don’t need to .read() the response and decode it yourelf. You can just use res.text, which will take care of applying appropriate decoders.

[PrimozGodec]

Yep, right now I am trying to make a standalone smallest example to reproduce the error.