gevent patch_all call in grequests breaks pymysql, etc.

I was looking forward to using this awesome package with my app and gevent until I noticed that it accomplishes gevent-compatibility via gevent.monkey.patch_all():

try:
    import gevent
    from gevent import monkey as curious_george
    from gevent.pool import Pool
except ImportError:
    raise RuntimeError('Gevent is required for grequests.')

# Monkey-patch.
curious_george.patch_all(thread=False, select=False)

Unfortunately, patch_all() impacts all other packages/modules used by the app, breaking some of them that were not designed for reentrancy. For example, gevent.monkey.patch_all() broke the combination of pymysql/pooled_db (with sporadic, hard-to-reproduce failures); my app has several greenlets that on occasion make pymysql calls; I isolated the problem to gevent’s monkey-patching of the socket module in that case.

A local solution involving monkey-patching of just the requests package to use gevent’s variants of blocking API’s would greatly enhance compatibility of grequests with apps and eliminate unexpected/undesirable side-effects on other packages. For example, I recently monkey-patched HBase/Thrift via the following code snippet with the desired effect on Thrift and no harm to the rest of the app (pymysql, etc.):

# Monkey-patch Thrift's TSocket to use gevent.socket to make our HBase interface
# gevent-friendly
import gevent.socket
from thrift.transport import TSocket
TSocket.socket = gevent.socket

gevent-zeromq is another example of local monkey-patching, albeit a more complex one: https://github.com/traviscline/gevent-zeromq/blob/master/gevent_zeromq/__init__.py

Thank you, Vitaly

[vitaly-krugl]

I find it really problematic for stability when an imported module all of a sudden changes so much of the environment for everything else that your app does. It’s akin to having the rug pulled from under your feet 😃

[vcarel]

I’m coming back on this issue, because gevent 1.1 introduced a warning message when monkey_patch is called multiple times:

grequests.py:21: RuntimeWarning: Patching more than once will result in the union of all True parameters being patched curious_george.patch_all(thread=False, select=False)

I don’t think libs should do monkey patch by themselves. If this is a requirement (and this is one obviously), I’d rather get a warning saying that “select”, “socket”, etc. should be monkey patched so that grequests can work. Regarding the above warning, imagine that there is another lib doing the same monkey patch. I would get the same warning! It doesn’t make sense…

Sure, it’s only a warning and it doesn’t hurt much. But nobody likes useless warnings, and I’m getting this one at every unit test, every run and in my production logs.

Please reconsider your decision to close this issue…