Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

'LifoQueue' object has no attribute 'mutex' when running gunicorn with eventlet and broker_pool_limit = 0

See original GitHub issue

I have this bug the first time I try to post a task with Celery from an API call:

Traceback (most recent call last):
  File "/home/fga/workspace/kombu_eventlet_bug/.virtualenv/lib/python3.4/site-packages/gunicorn/workers/async.py", line 56, in handle
    self.handle_request(listener_name, req, client, addr)
  File "/home/fga/workspace/kombu_eventlet_bug/.virtualenv/lib/python3.4/site-packages/gunicorn/workers/async.py", line 107, in handle_request
    respiter = self.wsgi(environ, resp.start_response)
  File "/home/fga/workspace/kombu_eventlet_bug/wsgi.py", line 18, in app
    add.delay(1,2)
  File "/home/fga/workspace/kombu_eventlet_bug/.virtualenv/lib/python3.4/site-packages/celery/app/task.py", line 412, in delay
    return self.apply_async(args, kwargs)
  File "/home/fga/workspace/kombu_eventlet_bug/.virtualenv/lib/python3.4/site-packages/celery/app/task.py", line 535, in apply_async
    **options
  File "/home/fga/workspace/kombu_eventlet_bug/.virtualenv/lib/python3.4/site-packages/celery/app/base.py", line 734, in send_task
    with self.producer_or_acquire(producer) as P:
  File "/home/fga/workspace/kombu_eventlet_bug/.virtualenv/lib/python3.4/site-packages/celery/app/base.py", line 863, in producer_or_acquire
    producer, self.producer_pool.acquire, block=True,
  File "/home/fga/workspace/kombu_eventlet_bug/.virtualenv/lib/python3.4/site-packages/celery/app/base.py", line 1232, in producer_pool
    return self.amqp.producer_pool
  File "/home/fga/workspace/kombu_eventlet_bug/.virtualenv/lib/python3.4/site-packages/celery/app/amqp.py", line 610, in producer_pool
    self._producer_pool.limit = self.app.pool.limit
  File "/home/fga/workspace/kombu_eventlet_bug/.virtualenv/lib/python3.4/site-packages/celery/app/base.py", line 1147, in pool
    pools.set_limit(limit)
  File "/home/fga/workspace/kombu_eventlet_bug/.virtualenv/lib/python3.4/site-packages/kombu/pools.py", line 137, in set_limit
    pool.resize(limit)
  File "/home/fga/workspace/kombu_eventlet_bug/.virtualenv/lib/python3.4/site-packages/kombu/resource.py", line 188, in resize
    self._shrink_down()
  File "/home/fga/workspace/kombu_eventlet_bug/.virtualenv/lib/python3.4/site-packages/kombu/resource.py", line 193, in _shrink_down
    with resource.mutex:
AttributeError: 'LifoQueue' object has no attribute 'mutex'

It seems to be because when gunicorn uses eventlet, the base class that is picked up is ‘eventlet.green.Queue.LifoQueue’ instead of the standard python one. It only happens if I set broker_pool_limit = 0.

I’ve set up a project to isolate and reproduce the bug: https://github.com/fgaudin/kombu_eventlet_bug