Cluster workers not being able to pick up Jobs / double pick up
See original GitHub issueThe following program will create two cluster
workers, then one job task. The job task will be marked as failed immediately. Every second the job is marked back as inactive
which should be picked up again by a single worker. However occasionally more than one worker is picking up the job cause it to be executed twice.
Is there something wrong with my logic or is this an issue with the cluster implementation in kue
?
var kue = require('kue');
var jobs = kue.createQueue();
var cluster = require('cluster');
if(cluster.isMaster){
cluster.fork();
cluster.fork();
jobs.create('test').save();
setInterval(function () {
kue.Job.rangeByState( 'failed', 0, -1, 'asc', function( err, jobs ) {
if(err){
console.log(err);
}
else{
jobs.forEach( function ( job ) {
job.state('inactive').save();
});
}
});
}, 1000);
}
if(cluster.isWorker){
jobs.process('test', function ( job, done ) {
console.log('worker' , cluster.worker.id + ' has control of: ' + job.id + ' | ' + new Date());
done('failed');
});
}
else{
jobs.on('job failed', function ( errorMessage ) {
console.log('Job failed');
})
}
When running the following program the jobs are eventually are not picked up at all.
Note: The done call is never called in this example which causes the state of the jobs to be changed from active
to inactive
. Instead of failed
to inactive
like the previous example.
var kue = require('kue');
var jobs = kue.createQueue();
var cluster = require('cluster');
if(cluster.isMaster){
cluster.fork();
cluster.fork();
jobs.create('test').save();
setInterval(function () {
kue.Job.rangeByState( 'active', 0, -1, 'asc', function( err, jobs ) {
if(err){
console.log(err);
}
else{
jobs.forEach( function ( job ) {
job.state('inactive').save();
});
}
});
}, 1000);
}
if(cluster.isWorker){
jobs.process('test', function ( job, done ) {
console.log('worker' , cluster.worker.id + ' has control of: ' + job.id + ' | ' + new Date());
});
}
else{
jobs.on('job failed', function ( errorMessage ) {
console.log('Job failed');
})
}
Issue Analytics
- State:
- Created 7 years ago
- Reactions:1
- Comments:10
Top Results From Across the Web
Cluster with 2 or 3 workers, jobs are not getting "distributed ...
However I always see the job being picked up by the first worker, the other one(s) aren't picking up the job. What am...
Read more >NodeJS Agenda scheduler: cluster with 2 or 3 workers, jobs ...
Now suppose I have 2 workers (processes) and I call "agenda.now()" from worker 1, then it can be picked up (processed) by any...
Read more >Best practices: Cluster configuration | Databricks on AWS
Learn best practices when creating and configuring Databricks clusters.
Read more >Best practices: Cluster configuration - Azure Databricks
Learn best practices when creating and configuring Azure Databricks clusters.
Read more >Fine Parallel Processing Using a Work Queue - Kubernetes
In this example, as each pod is created, it picks up one unit of ... to run this tutorial on a cluster with...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Hey @victornikitin,
Unfortunately I have not looked into this for more than 2 years. So I am not sure where Kue’s code stands nor if this is still an issue.