Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Couldn't apply scheduled task task_name: Channel.open: (506) No free channel ids

See original GitHub issue

version: python-celery-3.1.19 python-kombu-3.0.30

I noticed that Connect.ensure_connection would take no effect (no matter how many times) when virtual.Transport._avail_channel_ids exhausted already. (I presume this issue also exists in the latest version)

Can someone help me how to properly handle recovering from this scenario?

The try-except Exception in celery/beat.py Scheduler.apply_async is not enough to recover from this scenario.

File "/usr/lib/python2.7/site-packages/celery/bin/beat.py", line 79, in run
  return beat().run()
File "/usr/lib/python2.7/site-packages/celery/apps/beat.py", line 83, in run
  self.start_scheduler()
File "/usr/lib/python2.7/site-packages/celery/apps/beat.py", line 112, in start_scheduler
  beat.start()
File "/usr/lib/python2.7/site-packages/celery/beat.py", line 473, in start
  interval = self.scheduler.tick()
File "/usr/lib/python2.7/site-packages/celery/beat.py", line 221, in tick
  next_time_to_run = self.maybe_due(entry, self.publisher)
File "/usr/lib/python2.7/site-packages/celery/beat.py", line 207, in maybe_due
  exc, traceback.format_stack(), exc_info=True)

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/celery/beat.py", line 204, in maybe_due
    result = self.apply_async(entry, publisher=publisher)
  File "/usr/lib/python2.7/site-packages/celery/beat.py", line 260, in apply_async
    entry, exc=exc)), sys.exc_info()[2])
  File "/usr/lib/python2.7/site-packages/celery/beat.py", line 252, in apply_async
    **entry.options)
  File "/usr/lib/python2.7/site-packages/celery/app/task.py", line 560, in apply_async
    **dict(self._get_exec_options(), **options)
  File "/usr/lib/python2.7/site-packages/celery/app/base.py", line 354, in send_task
    reply_to=reply_to or self.oid, **options
  File "/usr/lib/python2.7/site-packages/celery/app/amqp.py", line 305, in publish_task
    **kwargs
  File "/usr/lib/python2.7/site-packages/kombu/messaging.py", line 172, in publish
    routing_key, mandatory, immediate, exchange, declare)
  File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 457, in _ensured
    interval_max)
  File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 369, in ensure_connection
    interval_start, interval_step, interval_max, callback)
  File "/usr/lib/python2.7/site-packages/kombu/utils/__init__.py", line 246, in retry_over_time
    return fun(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 237, in connect
    return self.connection
  File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 741, in connection
    self._connection = self._establish_connection()
  File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 696, in _establish_connection
    conn = self.transport.establish_connection()
  File "/usr/lib/python2.7/site-packages/kombu/transport/virtual/__init__.py", line 809, in establish_connection
    self._avail_channels.append(self.create_channel(self))
  File "/usr/lib/python2.7/site-packages/kombu/transport/virtual/__init__.py", line 791, in create_channel
    channel = self.Channel(connection)
  File "/usr/lib/python2.7/site-packages/kombu/transport/mongodb.py", line 92, in __init__
    super(Channel, self).__init__(*vargs, **kwargs)
  File "/usr/lib/python2.7/site-packages/kombu/transport/virtual/__init__.py", line 390, in __init__
    self.connection.channel_max), (20, 10),
SchedulingError: Couldn't apply scheduled task task_name: Channel.open: (506) No free channel ids, current=204, channel_max=65535

Issue Analytics

State:
Created a year ago
Comments:9 (5 by maintainers)

Top GitHub Comments

2reactions

thedrowcommented, Mar 22, 2022

@auvipy Please avoid closing issues without explaining why they were closed.

1reaction

chaneyzorncommented, Mar 23, 2022

@chaneyzorn Are you using celery 3.x?

@thedrow Yes，I’am using celery-3.1.19.

I just found a possible clear path to reproduce this problem with this version，and found a patch https://github.com/celery/kombu/pull/1470 may change the logic. (the change is not too long ago. I actually found it yesterday)

I am going to compare the impact of this patch before and after, and whether the problem still exists in the latest version.

More definitive results would be given after the experiment.

Top Results From Across the Web

Cannot use username variable inside scheduled task

But the problem is that when I test the command on the task scheduler locally, it seems no to accept the username as...

User GPP Scheduled Task item fails to apply - Windows Server

This article provides a solution to an issue where User Group Policy Preference (GPP) Scheduled Task item fails to apply.

Schtasks - Scheduled tasks - Windows CMD - SS64.com

Syntax: Create a new scheduled task: SCHTASKS /Create [Connect_Options] ... Show the security identifier corresponding to a scheduled task name: SCHTASKS ...

Cisco DNA Center User Guide, Release 2.3.4

In the Schedule Task window, click the toggle button to enable or disable Discover ... to switch to a new channel without any...

Windows Task Scheduler Fails

Possible reasons for this are: the profile name is wrong, the profile name is not wrapped in double-quotes, or you have scheduled under...