question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Celery integration captures retried exceptions that it shouldn't when one task directly calls another

See original GitHub issue

I have one Celery task that directly calls another, but as a function, not as a task. The first task is auto-retried for a certain set of exceptions, and the second task is retried for a different set of exceptions. Here is an example task in my actual codebase, but here is an isolated illustration of the problem.

Consider the following two tasks, bar which is auto-retried for exceptions A or B, and a wrapper task bat which is auto-retried for exceptions C or D.

def foo():
    ...  # do something that might raise exception A, B, C, or D

@app.task(autoretry_for=(A, B))
def bar():
    return foo()

@app.task(autoretry_for=(C, D))
def bat():
    return bar()

Now invoke bat:

bar.delay()

Now suppose that during the execution of bat, foo() raises exception C. Even though Celery will retry the task, the exception C will be reported to Sentry.

Issue Analytics

  • State:open
  • Created 4 years ago
  • Reactions:8
  • Comments:7 (3 by maintainers)

github_iconTop GitHub Comments

5reactions
antonpirkercommented, Feb 22, 2022

Quick update: This issue is still open. And because multiple persons are reporting this it will be kept in our minds for one of the next SDK updates.

0reactions
github-actions[bot]commented, Dec 23, 2021

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you label it Status: Backlog or Status: In Progress, I will leave it alone … forever!


“A weed is but an unloved flower.” ― Ella Wheeler Wilcox 🥀

Read more comments on GitHub >

github_iconTop Results From Across the Web

python - Why does Celery NOT throw an Exception when the ...
Going and checking the logs shows that the error IS getting thrown in the task (and if you want the message, ask), and...
Read more >
Celery in the wild: tips and tricks to run async tasks in the real ...
The thing about async tasks is that the hard part is not how to run ... Celery provides a retry method that can...
Read more >
Celery Documentation - Read the Docs
Celery is easy to integrate with web frameworks, ... Task.retry() call will raise an exception so any code after the retry won't be...
Read more >
Asynchronous tasks pattern
The primary problem we're trying to solve is to make it easy to move from a normal python function to an asynchronous task....
Read more >
5 tips for writing production-ready Celery tasks - Wolt Careers
While implementing a new (or modifying an existing) celery task, think about what can go south. If a task is calling some third-party...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found