[Q] How to process only one job and then shutdown?

If I run node worker.js then the process stays alive and waits for any jobs to process. And ofcourse it processes any jobs it can find … BUT

Is there a way to shutdown once a job is processed?

I know it sounds strange but for me this is important.

Can I hook into the done method somehow and pass it an anonymous function to execute a queue.shutdown?

Any ideas or solutions are welcome.

[pulkitsinghal]

I think the docker logs were confusing me.

the way container logs were being colored and spit out on my console, made it seem like one worker run was handling multiple jobs before shutting down sometimes.

This test shows a worker that handles multiple jobs where bad code leaks data across jobs.

worker_1    | f1a2e39eb955 1 6/3/2017, 5:08:02 PM doNotCorruptGlobalNamespace 2 <-- START
worker_1    | f1a2e39eb955 1 6/3/2017, 5:08:02 PM successfully ran the job: 549
worker_1    | f1a2e39eb955 1 6/3/2017, 5:08:02 PM inside queue.process("crawl") for job id: 550
worker_1    | f1a2e39eb955 1 6/3/2017, 5:08:02 PM inside doSomething()
worker_1    | f1a2e39eb955 1 6/3/2017, 5:08:07 PM doNotCorruptGlobalNamespace 3 <-- LEAK/CORRUPT
worker_1    | f1a2e39eb955 1 6/3/2017, 5:08:07 PM successfully ran the job: 550
worker_1    | f1a2e39eb955 1 6/3/2017, 5:08:08 PM inside queue.process("crawl") for job id: 551
worker_1    | f1a2e39eb955 1 6/3/2017, 5:08:08 PM inside doSomething()
worker_1    | f1a2e39eb955 1 6/3/2017, 5:08:13 PM doNotCorruptGlobalNamespace 4 <-- LEAK/CORRUPT
worker_1    | f1a2e39eb955 1 6/3/2017, 5:08:13 PM successfully ran the job: 551

This test proves that a worker which can properly shutdown after one job is finished, will not leak data across jobs.

worker_1    | 197f623cd9d9 1 6/3/2017, 5:49:49 PM inside queue.process("crawl") for job id: 570
worker_1    | 197f623cd9d9 1 6/3/2017, 5:49:49 PM inside doSomething()
worker_1    | 197f623cd9d9 1 6/3/2017, 5:49:54 PM doNotCorruptGlobalNamespace 2  <-- START
worker_1    | 197f623cd9d9 1 6/3/2017, 5:49:54 PM successfully ran the job: 570
worker_1    | 197f623cd9d9 1 6/3/2017, 5:49:55 PM [ All jobs finished. Kue is shut down. ]
worker_1    | 197f623cd9d9 1 6/3/2017, 5:49:59 PM [ Shutting down when all jobs finish... ]
worker_1    | 197f623cd9d9 1 6/3/2017, 5:49:59 PM inside queue.process("crawl") for job id: 571
worker_1    | 197f623cd9d9 1 6/3/2017, 5:49:59 PM inside doSomething()
worker_1    | 197f623cd9d9 1 6/3/2017, 5:50:04 PM doNotCorruptGlobalNamespace 2 <-- NO LEAKS
worker_1    | 197f623cd9d9 1 6/3/2017, 5:50:04 PM successfully ran the job: 571
worker_1    | 197f623cd9d9 1 6/3/2017, 5:50:04 PM [ All jobs finished. Kue is shut down. ]
worker_1    | 197f623cd9d9 1 6/3/2017, 5:50:08 PM [ Shutting down when all jobs finish... ]
worker_1    | 197f623cd9d9 1 6/3/2017, 5:50:08 PM inside queue.process("crawl") for job id: 572
worker_1    | 197f623cd9d9 1 6/3/2017, 5:50:08 PM inside doSomething()
worker_1    | 197f623cd9d9 1 6/3/2017, 5:50:13 PM doNotCorruptGlobalNamespace 2 <-- NO LEAKS
worker_1    | 197f623cd9d9 1 6/3/2017, 5:50:13 PM successfully ran the job: 572
worker_1    | 197f623cd9d9 1 6/3/2017, 5:50:13 PM [ All jobs finished. Kue is shut down. ]
worker_1    | 197f623cd9d9 1 6/3/2017, 5:50:20 PM [ Shutting down when all jobs finish... ]

Here is the smaller version of docker-compose.yml which I used to run tests:

version: '2'
services:
  worker:
    restart: always
    image: node:6
    environment:
      - REDIS_URL=${REDIS_URL}
    volumes:
      - ./worker/redisKueWorker.js:/apps/redisKueWorker.js
      - ./worker/node_modules:/apps/node_modules
    entrypoint:
      - node
      - /apps/redisKueWorker.js

[behrad]

not needed if you think so 😃