question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Worker doesn't start processing after reconnect

See original GitHub issue

I am getting following error lately:

You have triggered an unhandledRejection, you may have forgotten to catch a Promise rejection:
Error: Missing Job 17568662 when trying to move from active to delayed
     at Function.moveToDelayed (/usr/app/analytics-server/node_modules/bullmq/dist/classes/scripts.js:178:23)
     at runMicrotasks (<anonymous>)
     at processTicksAndRejections (internal/process/task_queues.js:97:5)
 exited with code [1] via signal [SIGINT]
 starting in -cluster mode-
 Listening for monitoring :4443
 Error: getaddrinfo ENOTFOUND private-db-redis-XXXXXXXXX.db.ondigitalocean.com
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:66:26) {
   errno: 'ENOTFOUND',
   code: 'ENOTFOUND',
   syscall: 'getaddrinfo',
   hostname: 'ENOTFOUND private-db-redis-XXXXXXXXX.db.ondigitalocean.com'
 }

After the container restarts it can’t connect to Redis anymore. Bull doesn’t even recognize that it can’t reconnect so the process stays alive.

What is the best way handling this?

  1. How can I make sure that there is no unhandledRejection in that case?
  2. Is there a way to reconnect a worker?

My healthcheck is calling queue.getWaitingCount() and this seems to work fine after the restart but the worker won’t start processing again

Thanks!

Version: bullmq@1.15.1

Issue Analytics

  • State:open
  • Created 2 years ago
  • Reactions:1
  • Comments:18 (9 by maintainers)

github_iconTop GitHub Comments

3reactions
eltoroitcommented, Apr 23, 2021

UPDATE It was my fault. I had a typo in the name of the QueueScheduler, the names for Worker, Queue and QueueScheduler must match.

I am testing the restart of the jobs by manually killing the process (CTRL+C) because my server will restart the job every day, I am doing a proper closing of the Queue, QueueScheduler, Worker, Redis. When I restart the process, I can query it in the active state but it does not seem to be re-processing the job.

Is there something else I have to do to get the jobs restarted?

1reaction
manastcommented, Apr 20, 2021

I cannot find anything wrong in BullMQ, but I wonder if you are attaching an error handler to your worker?, like:

myworker.on('error', (err) => console.error(err));

Because the way NodeJS works, if you are lacking such a listener, the process will exit with an error… (yes I also think this is an awkward behaviour).

Read more comments on GitHub >

github_iconTop Results From Across the Web

Anaconda worker could not start because: connection to ...
Anaconda worker could not start because: connection to localhost timed out after 0.2s. tried to connect 7 > times during 2.0 seconds.
Read more >
Kernel does not reconnect after disconnection #6910 - GitHub
I just tried this with jlab 1.0.3, FF 68, macOS. Here's what happened for me: Start up notebook server on port 8888; Start...
Read more >
The service worker lifecycle - web.dev
A service worker won't receive events like fetch and push until it successfully finishes installing and becomes "active".
Read more >
Troubleshoot refresh scenarios - Power BI - Microsoft Learn
This article provides ways to troubleshoot issues with refreshing data within the Power BI service, for various data sources and conditions.
Read more >
Implications of "Work from Anywhere" - When Remote Workers ...
Among the action steps suggested later will be a process to determine the impact of an employee working in a state in which...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found