Grequests use of `monkey.patch_all(...)` breaks my application

The aggressive python stdlib monkey patching originating from https://github.com/kennethreitz/grequests/blob/master/grequests.py#L21 causes a lot of breakage throughout my app. The degree of monkey patching causes the redis client to error out which in turn then breaks all async celery processing.

I really like the grequests lib, but cannot use it if it’s going to make things that used to work fine not work anymore.

  File "/Users/jay/my-app/venv/lib/python2.7/site-packages/billiard/pool.py", line 1426, in safe_apply_callback
    fun(*args)
  File "/Users/jay/my-app/venv/lib/python2.7/site-packages/celery/worker/job.py", line 347, in on_success
    self.acknowledge()
  File "/Users/jay/my-app/venv/lib/python2.7/site-packages/celery/worker/job.py", line 453, in acknowledge
    self.on_ack(logger, self.connection_errors)
  File "/Users/jay/my-app/venv/lib/python2.7/site-packages/kombu/transport/base.py", line 100, in ack_log_error
    self.ack()
  File "/Users/jay/my-app/venv/lib/python2.7/site-packages/kombu/transport/base.py", line 95, in ack
    self.channel.basic_ack(self.delivery_tag)
  File "/Users/jay/my-app/venv/lib/python2.7/site-packages/kombu/transport/virtual/__init__.py", line 511, in basic_ack
    self.qos.ack(delivery_tag)
  File "/Users/jay/my-app/venv/lib/python2.7/site-packages/kombu/transport/redis.py", line 127, in ack
    self._remove_from_indices(delivery_tag).execute()
  File "/Users/jay/my-app/venv/lib/python2.7/site-packages/redis/client.py", line 1919, in execute
    return execute(conn, stack, raise_on_error)
  File "/Users/jay/my-app/venv/lib/python2.7/site-packages/redis/client.py", line 1811, in _execute_transaction
    self.parse_response(connection, '_')
  File "/Users/jay/my-app/venv/lib/python2.7/site-packages/redis/client.py", line 1882, in parse_response
    self, connection, command_name, **options)
  File "/Users/jay/my-app/venv/lib/python2.7/site-packages/redis/client.py", line 387, in parse_response
    response = connection.read_response()
  File "/Users/jay/my-app/venv/lib/python2.7/site-packages/redis/connection.py", line 307, in read_response
    response = self._parser.read_response()
  File "/Users/jay/my-app/venv/lib/python2.7/site-packages/redis/connection.py", line 105, in read_response
    response = self.read()
  File "/Users/jay/my-app/venv/lib/python2.7/site-packages/redis/connection.py", line 90, in read
    return self._fp.readline()[:-2]
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 447, in readline
    data = self._sock.recv(self._rbufsize)
  File "/Users/jay/my-app/venv/lib/python2.7/site-packages/gevent/socket.py", line 392, in recv
    self._wait(self._read_event)
  File "/Users/jay/my-app/venv/lib/python2.7/site-packages/gevent/socket.py", line 298, in _wait
    self.hub.wait(watcher)
  File "/Users/jay/my-app/venv/lib/python2.7/site-packages/gevent/hub.py", line 341, in wait
    result = waiter.get()
  File "/Users/jay/my-app/venv/lib/python2.7/site-packages/gevent/hub.py", line 568, in get
    return self.hub.switch()
  File "/Users/jay/my-app/venv/lib/python2.7/site-packages/gevent/hub.py", line 331, in switch
    return greenlet.switch(self)
LoopExit: This operation would block forever

Additional references pertaining to gevent monkey patching issues:

• http://stackoverflow.com/questions/13040458/why-is-gevent-spawn-different-than-a-monkeypatched-threading-thread
• https://github.com/jrief/django-websocket-redis/issues/3
• http://bighow.net/5332403-Why_is__gevent_spawn__different_than_a_monkeypatched__threading_Thread____.html
• http://stackoverflow.com/questions/9192539/using-gevent-monkey-patching-with-threading-makes-thread-work-serially
• http://pastebin.com/TPYMvuVr

[lingxiaoyang]

I was having the exact same problem with celery(3.1.25)+redis: LoopExit: This operation would block forever.

My workaround inspired by @dmk23:

from gevent import monkey
import socket
import grequests
reload(socket)  # immediately restore the patch
.....
@task(...)
def my_celery_task(...):
    ......   # do some stuff
    # when I need grequests
    monkey.patch_socket()  # repatch
    ...
    # grequests has done its job
    reload(socket)
    .....  # continue to do other stuff

This workaround also solved the problem that Celery task status was not updated while running (no RECEIVED->STARTED), which is perhaps related to redis broker.

Later I had another problem with Google Search API due to the patched ssl module. I did the same: reload(ssl) and monkey.patch_ssl() then the error was fixed.

If you have similar problems, have a look at which ‘native’ modules it’s referencing in the stack trace. Doing a reload and repatch may solve these problems.

But still, I’m not sure if this is a community-verified solution. Perhaps a better solution lies in grequest itself: see #8 .

[nicbou]

@kennethreitz, why close the ticket without a solution? As far as I know, this is still an issue.