question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Timelimiter metrics not calculated when using resilince4j-kotlin

See original GitHub issue

Resilience4j version: 1.5.0

Kotlin version: 1.4.10

Timelimiter metrics are not calculated when using the resilience4j-kotlin module, on the Timelimiter.decorateSuspendFunction extension function.

Probably the counter resilience4j_timelimiter_calls_total should increment whenever a TimeoutCancellationException is thrown in the extension function, however its value remains 0.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:5 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
ZzAvecommented, Sep 29, 2020

The behaviour’s a little more fine fineTuned than I expected, although mostly related to kotlin’s way of dealing with CancellationExceptions

I’m wrapping my timeLimiter into a circuitbreaker, such that I can stop hammering an upstream service if it’s getting really slow.

circuitbreaker
   timelimiter
       <codeBlock calling an upstream service>

Now, if the timeLimiter returns in an error state (calling the the onError method), the circuitbreaker doesn’t pick up the underlying event as an Error as well, and there are no metrics on the event as well, which is something I don’t really get (yet). It seems that an error in terms of a timelimiter should never really occur?

Because of the intricate behaviour of kotlin’s TimeoutCancellationExceptions, for which I want to be quite explicit in my usage, I did the following:

suspend fun <T>  executeCircuitBreakingTimeLimitedFun(
    circuitBreaker: CircuitBreaker,
    timeLimiter: TimeLimiter,
    description: String,
    block: suspend () -> T
): T {
    val timeLimitedBlock = timeLimiter.decorateSuspendFunction(block)
    val circuitBreakingTimeLimitedBlock = circuitBreaker.decorateSuspendFunction {
        try {
            timeLimitedBlock()
        } catch (e: TimeoutCancellationException){
            // TimeoutCancellationExceptions are handled in a special way by kotlin coroutines and return a block value rather than a timeoutexception
            // https://github.com/Kotlin/kotlinx.coroutines/blob/1eeed509abbe8e542124841ce14884202463460b/kotlinx-coroutines-core/common/src/intrinsics/Undispatched.kt#L106
            throw TimeoutException(e.message ?: "", description)
        }
    }

        return circuitBreakingTimeLimitedBlock()
}

With that, both my circuitbreaker and timelimiter can works as expected, and all exceptions are recorded ‘properly’. Timeouts are registered as failed requests in my circuitbreaker, and requests that failed otherwise will both be registered in the timeLimiter metrics as well as the circuitbreaker metrics (which makes sense, since they’re nested). I’m not familiar with the approach in spring-boot with the aspectj annotations, but I’d expect a similar behaviour there.

The changes I’m making to resilience:

  • A timelimiter rethrows any exception
  • A timelimiter records TimeoutCancellationExceptions as java.util.concurrent.TimeoutExceptions, such that the event handler recognises this as a timeout.
  • A timelimiter records any other exception as an error.
  • A timelimiter records successfull results if no exception occured during execution
0reactions
RobWincommented, Sep 29, 2020

Yes, please

Read more comments on GitHub >

github_iconTop Results From Across the Web

Implementing Timeouts with Resilience4j - Reflectoring
Resilience4j's TimeLimiter can be used to set time limits (timeouts) on asynchronous operations implemented with CompleteableFuture s. The ...
Read more >
Micrometer - resilience4j
The following code snippet shows how to bind CircuitBreaker metrics to a MeterRegistry . It binds all CircuitBreaker instances at once and registers...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found