question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Messages to channels doesn't come after some channels in group

See original GitHub issue

I have a problem with message delivery to groups that have many channels in it.

I noticed that after I have many channels in a group – this channels stop receive messages posted to that group.

Here is my consumer class:

class NotificationConsumer(WebsocketConsumer):
    def connect(self):
        user = self.scope['user']
        if user.is_authenticated:
            async_to_sync(self.channel_layer.group_add)('auth', self.channel_name)
            async_to_sync(self.channel_layer.group_add)(user.channels_group_name, self.channel_name)
        else:
            async_to_sync(self.channel_layer.group_add)('noauth', self.channel_name)
        self.accept()

    def disconnect(self, close_code):
        user = self.scope['user']
        if user.is_authenticated:
            async_to_sync(self.channel_layer.group_discard)('user_%s' % user.id, self.channel_name)
            async_to_sync(self.channel_layer.group_discard)('auth', self.channel_name)  
        else:
            async_to_sync(self.channel_layer.group_discard)('noauth', self.channel_name)

    def message(self, event):
        message = event['message']
        self.send(text_data=json.dumps({'type': 'message', 'object': message}))

    def comment(self, event):
        comment = event['comment']
        status = event['status']
        self.send(text_data=json.dumps({
            'type': 'comment',
            'object': comment,
            'status': status,
        }))

Here is how I send a messages to the groups:

async_to_sync(channel_layer.group_send)('user_123', {'type': 'message', 'message': message_data})
async_to_sync(channel_layer.group_send)('auth', {'type': 'comment', 'comment': data, 'status': status})

Consumers, connected to the small personal user’s groups (called ‘user_[id]’) receiving messages is ok, while new connected consumers to the groups ‘auth’ and ‘noauth’ don’t receiving messages after some channels in the group (~200-300+ channels in the group I guess).

We use: channels==2.1.2, channels-redis==2.2.1, daphne==2.2.0, Django==2.0.6, Twisted==18.4.0

CHANNEL_LAYERS = {
    'default': {
        'BACKEND': 'channels_redis.core.RedisChannelLayer',
        'CONFIG': { "hosts": ["redis://127.0.0.1:6379/6"]},
    },
}

Channels run with daphne and Nginx in front.

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:9 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
andrewgodwincommented, Aug 7, 2018

In general I wouldn’t recommend Channels groups for any sort of large-scale work - you should probably roll your own solution if you have more than about 50 concurrent users in any single group, as you’ll know the best option to use at your scale.

Sorry we can’t make something that works better out of the box, but it would be a significant project to make them scale well to large group sizes and I don’t have a real-life deployment of that I can test on. Groups is more designed to have hundreds of groups with only a few members in each (i.e. per-user).

Andrew

On Mon, Aug 6, 2018 at 11:02 AM Dmitry notifications@github.com wrote:

Also I noticed that in log files we have this kind of warnings: WARNING Application instance <Task pending coro=<SessionMiddlewareInstance.call() running at /web/envs/project/lib/python3.6/site-packages/channels/sessions.py:175> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7fb4f03ecf48>()]>> for connection <WebSocketProtocol client=None path=b’/ws/notification/'> took too long to shut down and was killed.

This happens every time after sending message to auth and noauth groups.

Seems like Daphne kills process than take longer than 10 seconds. But it is strange for me that sending about 600-800 websocket messages takes so long. If I extend this Daphne timeout, for example, to 20 seconds than later users will receive messages in 20 seconds after event. I think it is too long for real time experience.

And also we faced with this error [Errno 24] Too many open files:

File “/web/envs/project/lib/python3.6/site-packages/channels/utils.py”, line 57, in await_many_dispatch await task File “/web/envs/project/lib/python3.6/site-packages/channels/utils.py”, line 49, in await_many_dispatch result = task.result() File “/web/envs/project/lib/python3.6/site-packages/channels_redis/core.py”, line 184, in receive self.receive_loop_task.result() File “/web/envs/project/lib/python3.6/site-packages/channels/sessions.py”, line 175, in call return await self.inner(receive, self.send) File “/web/envs/project/lib/python3.6/site-packages/channels/middleware.py”, line 41, in coroutine_call await inner_instance(receive, send) File “/web/envs/project/lib/python3.6/site-packages/channels/consumer.py”, line 54, in call await await_many_dispatch([receive, self.channel_receive], self.dispatch) File “/web/envs/project/lib/python3.6/site-packages/channels/utils.py”, line 57, in await_many_dispatch await task File “/web/envs/project/lib/python3.6/site-packages/channels/sessions.py”, line 175, in call return await self.inner(receive, self.send) File “/web/envs/project/lib/python3.6/site-packages/channels/middleware.py”, line 41, in coroutine_call await inner_instance(receive, send) File “/web/envs/project/lib/python3.6/site-packages/channels/consumer.py”, line 54, in call await await_many_dispatch([receive, self.channel_receive], self.dispatch) File “/web/envs/project/lib/python3.6/site-packages/channels/utils.py”, line 57, in await_many_dispatch await task File “/web/envs/project/lib/python3.6/site-packages/channels/utils.py”, line 49, in await_many_dispatch result = task.result() File “/web/envs/project/lib/python3.6/site-packages/channels/utils.py”, line 49, in await_many_dispatch result = task.result() File “/web/envs/project/lib/python3.6/site-packages/channels_redis/core.py”, line 184, in receive self.receive_loop_task.result() File “/web/envs/project/lib/python3.6/site-packages/channels_redis/core.py”, line 184, in receive self.receive_loop_task.result() File “/web/envs/project/lib/python3.6/site-packages/channels_redis/core.py”, line 202, in receive_loop real_channel, message = await self.receive_single(general_channel) File “/web/envs/project/lib/python3.6/site-packages/channels_redis/core.py”, line 218, in receive_single async with self.connection(index) as connection: File “/web/envs/project/lib/python3.6/site-packages/channels_redis/core.py”, line 447, in aenter self.conn = await aioredis.create_redis(**self.kwargs) File “/web/envs/project/lib/python3.6/site-packages/aioredis/commands/init.py”, line 177, in create_redis loop=loop) File “/web/envs/project/lib/python3.6/site-packages/aioredis/connection.py”, line 107, in create_connection timeout, loop=loop) File “/usr/lib/python3.6/asyncio/tasks.py”, line 339, in wait_for return (yield from fut) File “/web/envs/project/lib/python3.6/site-packages/aioredis/stream.py”, line 19, in open_connection lambda: protocol, host, port, **kwds) File “/usr/lib/python3.6/asyncio/base_events.py”, line 778, in create_connection raise exceptions[0] File “/usr/lib/python3.6/asyncio/base_events.py”, line 745, in create_connection sock = socket.socket(family=family, type=type, proto=proto) File “/usr/lib/python3.6/socket.py”, line 144, in init _socket.socket.init(self, family, type, proto, fileno) [Errno 24] Too many open files

Number of open files in system are significantly increased - 60-70k with sites running with Channels and Daphne and about 7-12k with our old uWSGI + Centrifugo deployment.

Unfortunately, we had to go back to the old proven solution (with uWSGI + Centrifugo) while we don’t solve these issues.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/django/channels/issues/1116#issuecomment-410797489, or mute the thread https://github.com/notifications/unsubscribe-auth/AACNVrZw_x8R8-crGkDsmI94HjR_NNSLks5uOISogaJpZM4VvJ10 .

0reactions
andrewgodwincommented, Aug 12, 2018

Closing as this isn’t an actionable issue any more (see http://channels.readthedocs.io/en/latest/support.html)

Read more comments on GitHub >

github_iconTop Results From Across the Web

Post a message to multiple channels - Microsoft Support
Select Post in multiple channels > Select channels. Choose which channels to post your message in. You can share wherever you're a member....
Read more >
Django channels Group send dies after sending a certain ...
This message will not display after a certain number of sends. I am doing some updating behavior where this send will occur every...
Read more >
React SDK does not show new group channels - JavaScript
For Sendbird, a channel with admin messages will be empty. Toni_U September 15, 2021, 11: ...
Read more >
Notify a channel or workspace - Slack
In channels with at least six members, Slack will ask you to confirm before you send a message with any of these mentions....
Read more >
How to Control Sending Email to Teams Channels
By default, any channel in a team can receive email from any sender. ... If you want the message to arrive in multiple...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found