Celery integration captures retried exceptions that it shouldn't when one task directly calls another
See original GitHub issueI have one Celery task that directly calls another, but as a function, not as a task. The first task is auto-retried for a certain set of exceptions, and the second task is retried for a different set of exceptions. Here is an example task in my actual codebase, but here is an isolated illustration of the problem.
Consider the following two tasks, bar
which is auto-retried for exceptions A
or B
, and a wrapper task bat
which is auto-retried for exceptions C
or D
.
def foo():
... # do something that might raise exception A, B, C, or D
@app.task(autoretry_for=(A, B))
def bar():
return foo()
@app.task(autoretry_for=(C, D))
def bat():
return bar()
Now invoke bat
:
bar.delay()
Now suppose that during the execution of bat
, foo()
raises exception C
. Even though Celery will retry the task, the exception C
will be reported to Sentry.
Issue Analytics
- State:
- Created 4 years ago
- Reactions:8
- Comments:7 (3 by maintainers)
Top Results From Across the Web
python - Why does Celery NOT throw an Exception when the ...
Going and checking the logs shows that the error IS getting thrown in the task (and if you want the message, ask), and...
Read more >Celery in the wild: tips and tricks to run async tasks in the real ...
The thing about async tasks is that the hard part is not how to run ... Celery provides a retry method that can...
Read more >Celery Documentation - Read the Docs
Celery is easy to integrate with web frameworks, ... Task.retry() call will raise an exception so any code after the retry won't be...
Read more >Asynchronous tasks pattern
The primary problem we're trying to solve is to make it easy to move from a normal python function to an asynchronous task....
Read more >5 tips for writing production-ready Celery tasks - Wolt Careers
While implementing a new (or modifying an existing) celery task, think about what can go south. If a task is calling some third-party...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Quick update: This issue is still open. And because multiple persons are reporting this it will be kept in our minds for one of the next SDK updates.
This issue has gone three weeks without activity. In another week, I will close it.
But! If you comment or otherwise update it, I will reset the clock, and if you label it
Status: Backlog
orStatus: In Progress
, I will leave it alone … forever!“A weed is but an unloved flower.” ― Ella Wheeler Wilcox 🥀