Using asgiref.local.Local with gevent leads to memory leaks

We use Django with Gunicorn and gevent workers. Since updating to Django 3, we started seeing some reasonably severe memory leaks in our web processes.

I bisected one of the leaks I could reproduce locally to https://github.com/django/django/commit/a415ce70bef6d91036b00dd2c8544aed7aeeaaed.

In particular, it seems to be caused by the switch from threading.local to asgiref.local.Local.

I investigated a fair bit. There seem to be a few things going on:

• this isinstance(key, threading.Thread) test evaluating to False

  (Note: else: is not handled there at all.)

• key.is_alive() also evaluating to False (this is after working around the previous problem)

(Also, I’m not sure how gevent-friendly CLEANUP_INTERVAL = 60 is.)

It could be argued that gevent-related bugs are a problem with gevent. However, I think this problem is quite severe and also not easily noticed as it’s mostly a silent problem.

While any fixes (here or in gevent) would be appreciated, it would be good to document the incompatibility too.

(I tested gevent 1.4.0 and 1.5a3.)

Thanks

[andrewgodwin]

That’ll be the problem then. Hard to reason about threads when they’re… not threads.

I’ll need to think about the best approach to this one. I really don’t want to have asgiref depend on gevent at all, but detecting that threads aren’t threads is going to be tricky otherwise.

[andrewgodwin]

Alright, I’ll get it merged in then, as it seems to solve at least two problems and not cause any issues!