CircuitBreaker state into redis or some external storage

Context:

• .NET 6
• ASP
• Polly 7.2.4

I’m trying to store breaker state into redis or some external storage. This would allow me to have 3 pods of my app while they share the same circuit breaker state.

I actually have a list of ad domain controllers, so if one of them fails i automatically retry on the another one. With something like this:

.Retry(_configuration.HostNames.Count, onRetry: (exception, retryCount, context) =>
{
   context[CONTEXT_KEY_URL] = _configuration.HostNames[retryCount];
});

Then when i have to execute i use something like this:

 _connectPolicy.Execute(
    action: context =>
    {
        var url = (string) context[CONTEXT_KEY_URL];
        var policyOfUrl = _policyPerHostname[url];
        
        policyOfUrl.Execute(() => connection.Connect((string)context[CONTEXT_KEY_URL], _configuration.Port));                    
    },
    context: new Context {
        [CONTEXT_KEY_URL] = _configuration.HostNames.First()
    }
);

The object _policyPerHostname is just a Dictionary<string, Policy> where the key is the single hostname. In this way i can store the local state on the breaker per each hostname, so if it fails i keep the fail and on every request that I recive it falls back almost instantly to the working hostname.

So my question is how can i store the state? My idea was to keep the service that stores thìs whole logic Scoped. Then every time that i initialize the service i fetch the state from redis and by using the hooks like onReset, onBreak, onHalfOpen i store the new state… But i would prefer something like more native support from the library.

[martincostello]

Hey there - we don’t expose the internal state of the circuit breakers, and we have no plans to do so. This use case isn’t something we would support in Polly.

I would recommend not attempting to do this and have each instance of your application have its own view of the state of the remote service with regards to the circuit breakers. For instance, if you had N > 1 instances and only a subset of them had a network routing issue that meant it could not successfully use the AD domain controller, then that degradation would spread to all of the instances even though they may not have the same problems.

[martincostello]

In that case I would stick with my original recommendation of not trying to synchronise the states between the instances.