Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Add exponential backoff with jitter option for webhook retries

See original GitHub issue

Currently, webhook retries are set to a fixed amount of time between retries. It would be great if there was an option to use capped exponential backoff and jitter for retries. The “Full Jitter” algorithm looks good, although “Decorrelated Jitter” also seems fine.

Full jitter:

temp = min(cap, base *2 ** attempt)
sleep = temp / 2 + random_between(0, temp / 2)

Decorrelated jitter:

sleep = min(cap, random_between(base, previous_sleep *3))

Some useful parameters to be able to control would be:

Maximum retry delay (cap) - the maximum amount of time to wait between attempts
Max attempts (perhaps, although I’d expect max retry delay to be easier to reason about in this situation)
Delivery timeout - the total amount of elapsed time we are willing to take before we consider the webhook delivery failed
Exponential rate (base) - base multiplier factor

Issue Analytics

State:
Created 5 years ago
Reactions:12
Comments:12 (4 by maintainers)

Top GitHub Comments

4reactions

danielcomptoncommented, Mar 15, 2019

This isn’t blocking us, it just seems like a better retry method to have instead of a fixed delay between webhook retries.

The Retry-After header will work in some cases, but to be able to respond with an exponential backoff here, the webhook needs to be up, and we need to store some state per webhook delivery. If we have both of those things, most of the time we will be able to process the webhook.

1reaction

Miaourtcommented, Jul 1, 2020

Excuse-me for the maybe noob question, but where is the published RFC we can comment ? I’m also quite interested by this feature.

Top Results From Across the Web

Better Retries with Exponential Backoff and Jitter

In this tutorial, we'll explore how we can improve client retries with two different strategies: exponential backoff and jitter.

Performance of WebFlux with retry exponential backoff

In my opinion these two approaches can be used together. The retry strategy is the simplest way to manage very transient error.

Handling failed webhooks with Exponential Backoff

In this post, I would like to introduce you to an approach used at Gympass to handle failed webhook requests to our partners,...

Timeouts, retries and backoff with jitter

Instead of retrying immediately and aggressively, the client waits some amount of time between tries. The most common pattern is an exponential backoff,...

Registration Postback Guide | SCORM Cloud Documentation

After each unsuccessful attempt, we add a delay before the next retry. That delay follows an exponential backoff with jitter algorithm.