Raised ConnectionClosedError with message "Reader at end of file"

Hi,

I’m currently seeing a lot of such errors in my Daphne logs in production. I’m not fully sure if this is a generic Channels issue, a Daphne one, or more specific to channels_redis.

Exception inside application: Reader at end of file
  File "/usr/local/lib/python3.7/site-packages/channels/sessions.py", line 183, in __call__
    return await self.inner(receive, self.send)
  File "/usr/local/lib/python3.7/site-packages/channels/middleware.py", line 41, in coroutine_call
    await inner_instance(receive, send)
  File "/usr/local/lib/python3.7/site-packages/channels/consumer.py", line 59, in __call__
    [receive, self.channel_receive], self.dispatch
  File "/usr/local/lib/python3.7/site-packages/channels/utils.py", line 52, in await_many_dispatch
    await dispatch(result)
  File "/usr/local/lib/python3.7/site-packages/asgiref/sync.py", line 150, in __call__
    return await asyncio.wait_for(future, timeout=None)
  File "/usr/local/lib/python3.7/asyncio/tasks.py", line 388, in wait_for
    return await fut
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.7/site-packages/channels/db.py", line 14, in thread_handler
    return super().thread_handler(loop, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/asgiref/sync.py", line 169, in thread_handler
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/channels/consumer.py", line 105, in dispatch
    handler(message)
  File "/usr/local/lib/python3.7/site-packages/channels/generic/websocket.py", line 106, in websocket_disconnect
    self.disconnect(message["code"])
  File "/project/platform/project/websockets/consumers.py", line 122, in disconnect
    async_to_sync(self.channel_layer.group_discard)(self.get_client_group(), self.channel_name)
  File "/usr/local/lib/python3.7/site-packages/asgiref/sync.py", line 79, in __call__
    return call_result.result()
  File "/usr/local/lib/python3.7/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/usr/local/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.7/site-packages/asgiref/sync.py", line 95, in main_wrap
    result = await self.awaitable(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/channels_redis/core.py", line 599, in group_discard
    await connection.zrem(key, channel)
  File "/usr/local/lib/python3.7/site-packages/aioredis/commands/sorted_set.py", line 224, in zrem
    return self.execute(b'ZREM', key, member, *members)
  File "/usr/local/lib/python3.7/site-packages/aioredis/commands/__init__.py", line 51, in execute
    return self._pool_or_conn.execute(command, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/aioredis/connection.py", line 319, in execute
    raise ConnectionClosedError(msg)
  Reader at end of file

The code referenced to /project/platform/project/websockets/consumers.py is:

class WSConsumer(JsonWebsocketConsumer):
    http_user_and_session = True

    def connect(self):
        if not self.scope["user"].is_authenticated:
            self.close(code="4000")
        else:
            
            async_to_sync(self.channel_layer.group_add)("user-broadcast", self.channel_name)
            super().connect()

    def disconnect(self, code):
        if self.scope["user"].is_authenticated:
            # NEXT IS LINE 122 as referenced in the error stack
            async_to_sync(self.channel_layer.group_discard)("user-broadcast", self.channel_name)
        
        super().disconnect(code)

    def ws_send(self, event):
        self.send_json(event["body"])

I’m running Daphne from supervisord behind nginx. Both nginx and Daphne are in two different Docker containers running on Kubernetes. Redis is on a separate machine (the connection is over TLS).

Python: 3.7.3 Django: 2.1.8 Daphne: 2.3.0 Channels: 2.2.0 channels_redis: 2.4.0 aioredis: 1.2.0 Redis: 5.0.5 OS: Alpine Linux 3.9 (Docker image)

In channels_redis a similar issue has been reported but discarded: https://github.com/django/channels_redis/issues/158

Any hint on what could be the root cause for this issue?

Thanks for your time.

[RealScatman]

I use Heroku to host my application, and the redis server provisioned via Heroku sets a timeout of 300 seconds (5 minutes.) I set the timeout to 0 yesterday, and have yet to experience the issue again…hope this may help some people. You can also declare the maxmemory config option as well.

[RealScatman]

Hi all, I am also experiencing this issue in uat environment but not locally. Anyone have any ideas?