Concurrency issue in Caching decorator

Resilience4j version: 1.7.1

Java version: 8.0.345

We encountered a concurrency issue in Resilience4J: The implementation of the io.github.resilience4j.cache.Cache#computeIfAbsent might invoke the given Callable multiple times.

A better behavior would be to add some kind of locking, so that only the first invocation gets executed, but all following wait for the initial result to be served by the cache.

As a side note (potentially not relevant): We use caffeine in version 2.9.3 as the JCache implementation.

package my.sandbox;

import io.github.resilience4j.cache.Cache;
import io.vavr.CheckedFunction1;
import org.junit.Test;

import javax.cache.CacheManager;
import javax.cache.Caching;
import javax.cache.configuration.MutableConfiguration;
import javax.cache.spi.CachingProvider;
import java.util.concurrent.Callable;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.function.Supplier;

import static org.assertj.core.api.Assertions.assertThat;

public class NewTest {
    @Test
    public void testSomething()
            throws Exception
    {
        // Setting up the JCache
        final CachingProvider provider = Caching.getCachingProvider();
        final CacheManager manager = provider.getCacheManager();
        final MutableConfiguration<String, Integer> config = new MutableConfiguration<>();
        final javax.cache.Cache<String, Integer> javaxCache = manager.createCache("myCache", config);

        // Setting up the Resilience4J Cache
        final Cache<String, Integer> resilience4JCache = Cache.of(javaxCache);

        // Decorate the callable and run in different Threads
        final CheckedFunction1<String, Integer> decoratedCall = Cache.decorateCallable(resilience4JCache, new IntegerCallable());
        final CompletableFuture<Integer> soonResult1 = CompletableFuture.supplyAsync(new SupplierAdapter<>(decoratedCall));
        final CompletableFuture<Integer> soonResult2 = CompletableFuture.supplyAsync(new SupplierAdapter<>(decoratedCall));

        // Wait for result and check that it's the same/different
        final Integer result1 = soonResult1.get();
        final Integer result2 = soonResult2.get();
        assertThat(result1).isEqualTo(result2);
    }

    
    //Small adapter to use a checked function as a Supplier.
    static class SupplierAdapter<V> implements Supplier<V> {

        private final CheckedFunction1<String, V> checkedFunction;

        SupplierAdapter(CheckedFunction1<String, V> callable) {
            this.checkedFunction = callable;
        }

        @Override
        public V get() {
            try {
                return checkedFunction.apply("someKey");
            } catch (final Throwable e) {
                throw new RuntimeException(e);
            }
        }
    }

    // Callable expecting two calls to wait for each other before returning.
    // Just used to show the problem. Cannot be used to as a positive test, as it runs into a deadlock if the cache does not evaluate multiple times.
    private static class IntegerCallable implements Callable<Integer> {
        private static final CountDownLatch latch = new CountDownLatch(2);
        final AtomicInteger counter = new AtomicInteger(0);

        @Override
        public Integer call() throws Exception {
            final int newValue = counter.addAndGet(1);
            latch.countDown();
            latch.await();
            return newValue;
        }
    }
}

[RobWin]

PRs are very welcome. I won’t have to time to contribute a fix.

[cschubertcs]

Sorry for the long delay! We were investigating the EntryProcessor approach you proposed, to understand all implications.

As we have long-running Supplier functions, for example making multiple HTTP calls to systems that might be slow to respond (hence the caching) we run with all standard Cache approaches into the same problem: The Cache Eviction logic waits for the last computation of such a long-running operation, while not allowing new locks on the cache. This basically locks the whole cache for a potentially long time. This happens at least with Caffeine JCache implementation, but most likely also others, as the JCache spec does not provice any guarantees here, as you already described.

For our code we will therefore use an approach like this:

import java.util.concurrent.locks.ReentrantLock;

class CacheConsumer<K, V> {
    // cache eviction should be adjusted based on use case
    static final Cache<K, Lock> lockCache = Caffeine.newBuilder().build();

    V operationToBeCached() {
        // here some long-running operation is going on
    }

    void methodUsingCaching() {
        Cache jCache = getJCache();
        K cacheKey = getCacheKey();

        Lock lock = lockCache.get(cacheKey, l -> new ReentrantLock());
        try {
            lock.lock();

            T value = jCache.getIfPresent(cacheKey);
            if (value != null) {
                return value;
            }
            T newValue = operationToBeCached();
            jCache.put(cacheKey, newValue);
            return newValue;
        } finally {
            lock.unlock();
        }
    }
}

That way the calculation happens outside of the Cache, being sure that it does not block the cache processes.

Having said that, the solution with the EntryProcessor looks like a valid solution for the initial bug for use-cases with not-so-long-running operations. I would be happy to implement the necessary change! Even though the code is already provided by you I can at least try to write a test for this.