Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Can't call send on channel layer from celery task

See original GitHub issue

In continuation of this closed issue: https://github.com/django/channels/issues/1017

Trying to call async_to_sync(chanel_layer.send) from celery task causes an error:

You cannot use AsyncToSync in the same thread as an async event loop

channels 3.0.4 and daphne 3.02, so the issue wasn’t fixed, as opposed to what’s said in that thread

Issue Analytics

State:
Created 2 years ago
Reactions:2
Comments:6 (1 by maintainers)

Top GitHub Comments

1reaction

udbhavcommented, Apr 17, 2022

After investigating more, I think it might actually be something happening with channels in conjunction with gevent, I added a new test to the repo with a celery task that doesn’t call channels, and instead detects whether an event loop is running, and no event loop is created. However, as soon as I add a async_to_sync(channel_layer.group_send) call, running multiple tasks will trigger the runtime error. My fairly uneducated guess is that asgiref creates temporary event loops when calling async_to_sync, and these event loops occasionally aren’t fully closed when gevent is managing multiple greenlets.

EDIT: Experimenting more with my test case repo, there are no problems in python 3.6, issue only starts happening if I use python 3.7. Issue also does’t happen with prefork celery pool, but does with gevent. For my case, I’m going to create separate tasks that call channels’ group_send that will run in their own prefork worker pool.

0reactions

WYmindskycommented, Nov 2, 2022

After investigating more, I think it might actually be something happening with channels in conjunction with gevent, I added a new test to the repo with a celery task that doesn’t call channels, and instead detects whether an event loop is running, and no event loop is created. However, as soon as I add a async_to_sync(channel_layer.group_send) call, running multiple tasks will trigger the runtime error. My fairly uneducated guess is that asgiref creates temporary event loops when calling async_to_sync, and these event loops occasionally aren’t fully closed when gevent is managing multiple greenlets.

EDIT: Experimenting more with my test case repo, there are no problems in python 3.6, issue only starts happening if I use python 3.7. Issue also does’t happen with prefork celery pool, but does with gevent. For my case, I’m going to create separate tasks that call channels’ group_send that will run in their own prefork worker pool.

tks, python3.6 is ok, python3.7 cause the problem.

Top Results From Across the Web

python - Celery task can not call Channels consumer via ...

Celery task can not call Channels consumer via channel_layer.send ; sync_send(channelName, consumer, value): ; "type": consumer, "value" ; def ...

[Solved]-Celery task can not call Channels consumer via ...

Coding example for the question Celery task can not call Channels consumer via channel_layer.send-django.

Calling Tasks — Celery 5.2.7 documentation

This document describes Celery's uniform “Calling API” used by task instances ... Shortcut to send a task message, but doesn't support execution options....

Channel Layers — Channels 4.0.0 documentation

The channel layer is for high-level application-to-application communication. When you send a message, it is received by the consumers listening to the group...

Django, Channels, and Celery: How to ensure performance?

The user sends the pk to the channel name, and a Celery task is started which ... New comments cannot be posted and...