Inspect exception or return value in wait_gen (or wait_gen override)
See original GitHub issueI wanted to see if there is interest in an API addition to allow users to determine wait time based on exception or return value from the decorated function.
One use case being: You’re sending requests to a rate limited API. API blocks your request but is nice enough to include Retry-After
header in the response.
Retry-After
contains the exact number of seconds you should wait before making another request.
If I could access the exception or response somehow before wait time is decided, I could check for this header and if present use its value, otherwise fall back on a wait_gen
Using any kind of calculated time function is very hit and miss in this scenario and mostly wasteful.
I’m thinking about adding an optional wait_override
argument to the decorators. This would be a function that gets passed the exception (for backoff.on_exception
) or the retry-able’s return value (for backoff.on_predicate
).
So that users can do something like this:
def my_override(exception):
headers = getattr(e, 'headers', {})
seconds = headers.get('Retry-After')
# If None is returned, wait_gen takes over
return int(seconds) if seconds else None
@backoff.on_exception(backoff.expo, HTTPError, wait_override=my_override)
def request_thing(url):
response = do_request(...)
response.raise_for_status()
return response
Issue Analytics
- State:
- Created 6 years ago
- Comments:5 (4 by maintainers)
Top GitHub Comments
@edgarrmondragon Yes. Thanks. And released in latest backoff (2.0.1)
is this closed by https://github.com/litl/backoff/commit/ed932f749b9c6cfe1e374662c23bfaadcb8d12bc?