question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Task lock exceptions storing within huey.results.huey?

See original GitHub issue

Charles @coleifer, great repo, management and continued development.

I am running 1.6.1 and have recently leveraged the @huey.lock_task(<lock_name>); working well and as expected. One finding/question I did have was around the TaskLockedException. I have noticed that when the TaskLockedException is thrown, a result (exception and stack) is added to the huey.results.huey redis hash. My RedisHuey instantiation is RedisHuey(events=False, store_errors=False, store_none=False) and my function is return None. What I find is that, in instances of lock unavailability, this huey.results.huey just continues to populate with TaskLockedException entries; one per attempt no greater. And while a redis hash can get VERY large these “results” are not something I require. I am interested in knowing if this functionality is by design or if there is something I am missing from an implementation perspective. Thanks!

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Comments:7 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
coleifercommented, Feb 7, 2018

Proposed change:

--- a/huey/api.py
+++ b/huey/api.py
@@ -356,13 +356,12 @@ class Huey(object):
         try:
             result = task.execute()
         except Exception as exc:
-            if self.result_store:
+            if self.result_store and self.store_errors:
                 metadata = self._get_task_metadata(task, True)
                 metadata['error'] = repr(exc)
                 metadata['traceback'] = traceback.format_exc()
                 self.put(task.task_id, Error(metadata))
-                if self.store_errors:
-                    self.put_error(metadata)
+                self.put_error(metadata)
             raise
 
         if self.result_store and not isinstance(task, PeriodicQueueTask):

1reaction
coleifercommented, Feb 7, 2018

So, here’s the relevant code (api.py):

    def execute(self, task):
        if not isinstance(task, QueueTask):
            raise TypeError('Unknown object: %s' % task)

        try:
            result = task.execute()
        except Exception as exc:
            if self.result_store:
                metadata = self._get_task_metadata(task, True)
                metadata['error'] = repr(exc)
                metadata['traceback'] = traceback.format_exc()
                self.put(task.task_id, Error(metadata))
                if self.store_errors:
                    self.put_error(metadata)
            raise

So we’re storing errors as results, but I think the fix would be to move the “store_errors” conditional up to the line reading “if self.result_store”.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Huey's API — huey 2.4.4 documentation - Read the Docs
RedisExpireHuey uses a different approach: task results are stored in ordinary Redis keys with a ... Lock instead of a lockfile for file-system...
Read more >
huey Documentation - Read the Docs
If the caller resolves the Result now, a TaskException will be raised which contains information about the exception that occurred in our task....
Read more >
Retry a Huey task when something fails - Stack Overflow
My question is: Is there any way, that we can configure this Huey task to retry the execution. Let's say, retry 5 times...
Read more >
Huey task queue 2.0 : r/Python - Reddit
automatically retry tasks that fail. task prioritization (new). result storage. task locking. task pipelines and chains. optional django ...
Read more >
huey - PyPI
schedule recurring tasks, like a crontab. automatically retry tasks that fail. task prioritization. task result storage. task expiration. task locking.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found