Background Refresh / Refresh Ahead Policy

Current semantic

When enabling background refresh with CacheBuilder.refreshAhead(true):

The old value will be returned by the cache, although it is expired, and will be replaced by the new value, once the loader is finished. In the case there are not enough threads available to start the loading, the entry will expire immediately and the next get() request will trigger the load.

Once refreshed, the entry is in a trail period. If it is not accessed until the next expiry, no refresh will be done and the entry expires regularly.

Analysis

We have indication from our applications, that this is not working well in all scenarios. Example:

• expiry time is 5 minutes
• entry is accessed every 30 minutes on average
• Load has 3s latency

This means the second load is mostly useless and the cache has always a miss, when the entry is accessed by the application.

How long the refreshing is done should be separated from the normal expiry duration. Especially at nighttime the access frequency gets lower. Administrators should be able to decide between:

• minimize user latency, which means more refreshing
• minimize the memory footprint and loader calls, which means more latency for the user

Todo

• in-depth analysis of current cache scenarios
• what parameters make sense?
• what is the best behavior for backgroundRefresh(true), for the ease of use principle we want to have a setting which is a good fit for most of the time

Any more thoughts?

[cruftex]

Should be merged with https://github.com/cache2k/cache2k/issues/172. If we change to another mechanism to track the access, we could also recognize a second access after the initial load. This enables the possibility to do no refresh if the value was never accessed a second time.

[denghongcai]

Sorry for my delay.

What is the allowed stale time or how much ahead of expiry would you typically do the refresh?

typically, less than a minute

How long does a refresh take?

it depends, typically less than ten seconds. i think it’s not a major problem here.

What are the actual business requirements you try to implement (e.g. data may not be older than…)?

data must be always fresh if it is warm. we had a DataGateway to server http request, it read a expensive config from somewhere then use it to get/parse/edit from backend service. you can image it like a GraphQL Gateway instead of it’s a heavy operation on get GraphQL-ish DSL.