Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
Support for backup requests / hedge requests
See original GitHub issueSummary
Add support to linkerd to enable backup requests also known at hedge requests.
Context
Backup requests can be used to reduce the overall latency of a system, specifically tail latency. If a request starts to take longer than the n-th percentile, you can choose to issue the same request again, and use the response of the request which replies first to send to the client. This may work well for read requests without requiring modification of downstream systems.
Further reading and context:
- Software techniques that tolerate latency variability are vital to building responsive large-scale Web services
- The Tail at Scale discusses the above article
- Finagle already has implemented a BackupRequestFilter
- Simple implementation of hedge requests in go: hedged
Possible Implementation
I’ve never used linkerd before so this might be way off!
Looking at the configuration documenation it seems like retires might be a good place to configure backup requests. Whilst a backup request isn’t exactly a ‘retry’ it feels close enough?
Retry backup request
| Key | Default Value | Description |
|---|---|---|
| quantile | 95 |
The response latency quantile, above which a backup request will be issued. Must be between 0 and 100, exclusive. |
| clipDurationSec | 60 |
The maximum duration to measure. Used to create the latency histogram. Requests that take longer than clipDurationSec will be added to the histogram using this upper bound. |
| historySec | 3600 |
The length of time to keep latency information, used when calculating the quantile |
Tasks
- Decide on implementation
- Create pull request
- Code review
Issue Analytics
- State:
- Created 7 years ago
- Reactions:5
- Comments:7 (2 by maintainers)
Top Results From Across the Web
Top Related Medium Post
Top Related StackOverflow Question
Troubleshoot Live Code
Top Related Reddit Thread
Top Related Hackernoon Post
Top Related Tweet
Top Related Dev.to Post
Top Related Hashnode Post
@amitsaha as @jacktuck mentioned I’m not sure this approach would work for apis that are not idempotent. Perhaps we could include a response classifier section in the configuration similar to https://linkerd.io/config/1.3.5/linkerd/index.html#http-response-classifiers, this then allows the linkerd user to determine which types of requests can be hedged
This is a great request. Something I’ve been thinking about as well. Would be an awesome feature.