scheduler creating duplicate tasks in multiple cluster environment

We have a service that uses django-q for asynchronous tasks and it’s deployed as 2 instances (2 AWS EC2 servers each running the same django project and each running a django-q cluster to process tasks). We’ve encountered an issue where the same scheduled task – scheduled to run once – gets picked up by each of the clusters in the scheduler (django-q.cluster) and ends up having 2 separate tasks being created.

Example entries in our logs:

On server 1:

2017-04-02 20:25:56,747 - django-q - INFO - Process-1 created a task from schedule [14789]
2017-04-02 20:25:56,842 - django-q - DEBUG - Pushed ('hamper-india-magnesium-pip', 'f1a1141c1835400ebc4f4b3894922b82')

On server 2:

2017-04-02 20:25:56,853 - django-q - INFO - Process-1 created a task from schedule [14789]
2017-04-02 20:25:56,990 - django-q - DEBUG - Pushed ('alpha-william-kansas-apart', '5a4fcadb47674590933415dd5a71e1cc')

Is this the expected behavior or is it a bug?

What we are looking for is to have a scheduled task create only one async task to execute the action, even in a multi-cluster setup like ours.

Can you comment on this behavior?

We’re using:

Django (1.10.6)
django-q (0.7.18)

Thanks

-Kevin

[marcelolima]

I’m also facing the same issue and disabling the schedule on all but one machine is not easy in my case because of auto-scaling. Is there any update on this issue other than the suggested workaround? I noticed there is this PR https://github.com/Koed00/django-q/pull/347 that seems to fix the issue, but still in open.

[kefin]

Thank you @Koed00 for merging PR #347.