Getting an incorrect value back when the cache reloads.

I am using Guava cache with refreshAfterWrite(). I have set the TTL to 60 sec.

        private LoadingCache<ServiceCacheLoader.ServiceKey, Optional<Service>> getOrCreateIfAbsent(CacheConfiguration cacheConfiguration,
                CacheLoader<ServiceCacheLoader.ServiceKey, Optional<Service>> cacheLoader) {
            assert cacheConfiguration != null : "cacheConfiguration:null";
            assert cacheLoader != null : "cacheLoader:null";

            LoadingCache<ServiceCacheLoader.ServiceKey, Optional<Service>> cache = cacheInstance.get();
            if (cache == null) {
                cache = CacheBuilder.newBuilder()
                        .refreshAfterWrite(cacheConfiguration.getTimeToLiveInSeconds(), TimeUnit.SECONDS) // 60 seconds
                        .maximumSize(cacheConfiguration.getMaximumSize())
                        .concurrencyLevel(cacheConfiguration.getConcurrencyLevel()).build(cacheLoader);
                cacheInstance.compareAndSet(null, cache);
            }
            return cache;
        }

I am setting the key of type ServiceKey object.

public final class ServiceKey {

    private final String authority; // not unique
    private final String serviceKey; // not unique
    private final String consumerKey; // unique 

    // ..other fields like setter getters, hashcode, and equals
}

private static final Executor BACKGROUND_CACHE_EXECUTOR = MoreExecutors
        .getExitingExecutorService(new ThreadPoolExecutor(1, 5, 30, TimeUnit.SECONDS,
                new LinkedBlockingQueue<Runnable>())); // set on class level
    
public ListenableFuture<Optional<Service>> reload(final ServiceKey serviceKey,
            final Optional<Service> previousValue) {
        final ListenableFutureTask<Optional<Service>> task = ListenableFutureTask
                .create(new Callable<Optional<Service>>() {
                    @Override
                    public Optional<Service> call() throws Exception {
                        try {
                            return load(serviceKey);
                        } catch (final Exception ex) {
                            ERROR_REPORTER.report(ServiceCacheLoader.class.getName() + ".error.loadingService", ex,
                                    serviceKey, ex.getMessage(), previousValue.orNull());
                            throw new RuntimeException(ex);
                        }
                    }
                });
        BACKGROUND_CACHE_EXECUTOR.execute(task);
        return task;
    }

I am seeing a strange behavior whenever cache expires and the reload function is called. On reload I am getting the wrong previous value for the key I am currently querying the cache with. Let’s say my cache has Key 1 -> val1 and Key2 -> val2. On reload I get Key1 -> val2 back. Can someone help me understand this behavior? Give me some pointers on why this could be happening? This issue doesn’t happen if I use expireAfterWrite().

My suspicion is something funky going on with threading though I can’t pinpoint what. I am able to reproduce the issue by doing the following steps in server debug mode.

1. Make call1 with authority (“dev”), serviceKey (“v1”), and consumerKey(“1234”). Entry is created in cache and I get back results successfully.
2. Make call2 with authority (“dev”), serviceKey (“v1”), and consumerKey(“4567”). Entry is created in cache and I get back results successfully.
3. Make call1 again. Entry is expired by now so reload method is called. This in turn calls the load method and I get back results successfully.
4. Make call2 again. Entry is expired by now so reload method is called. This in turn calls the load method and I get back results successfully.
5. Make call2 again. Call fails because now cache returns call1 value. So confused on this step.

[ben-manes]

The problem in your fix is that it creates a new thread per call which is never stopped, causing a memory leak and no reuse. It sounds like you don’t manage request-scoped / thread-local state cleanly which causes a data leak. If you did this approach then I would use an explicit Thread which will exit when the task is done. Ideally the data leak should be fixed, though.

[ben-manes]

yes, that looks good. Is your cache loader dependent on other mutable data? Typically it is that something else is not synchronized correctly and causing data races. Ideally the cache loader is a idempotent, e.g. just runs a SQL query, but you might be calling other internal services that are not thread safe. If I was in your shoes then I would add logging and asserts to track down the problem.