Documentation about greenlets with djhuey

Hi,

When trying to make use of greenlets workers, the consumer task just doesn’t work. I tried to monkey patch using gevent, but it seems the worker thread is not able to connect to DB.

Settings

HUEY = {
    'connection': {'host': self.HUEY_HOST, 'port': self.HUEY_PORT},
}

Additional run_huey_greenlet management command :

from huey.contrib.djhuey.management.commands.run_huey import Command as RunHueyCommand
from django.conf import settings


class Command(RunHueyCommand):

    def __init__(self, *args, **kwargs):
        if settings.HUEY.get('consumer', {}).get('worker_type') == 'greenlet':
            from gevent import monkey
            monkey.patch_all()
        super(Command, self).__init__(*args, **kwargs)

CLI

./manage.py run_huey_greenlet

DatabaseError: DatabaseWrapper objects created in a thread can only be used in that same thread. The object with alias 'default' was created in thread id 139937579968256 and this is thread id 139937120251248.

[antwan]

It worked when patching the manage.py bootstrap script, as this is where the internal DB connection is made. Patching the actual command file doesn’t work as it’s too late.

#!/usr/bin/env python
import os
import sys

if 'run_huey' in sys.argv:
    from gevent import monkey
    monkey.patch_all()

if __name__ == "__main__":
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "conf")
    from django.core.management import execute_from_command_line
    execute_from_command_line(sys.argv)

I think it’s worth mentioning that in the doc + troubleshooting, because :

1. It’s not optional as stated, it’s a mandatory thing to do.
2. Most of the people use djhuey and there is no mention anywhere about how to do it properly using django.

[coleifer]

Added docs: http://huey.readthedocs.io/en/latest/contrib.html#using-gevent