Allow returning stale values while caches are revalidated

TLDR: Support a staleWhileRevalidate hint on the cacheControl directive

type Variant {
  name: String
  sku: String
  inStock: Boolean @cacheControl(maxAge: 5, staleWhileRevalidate: 100)
}

I’ve been experimenting with cacheControl hints on an Apollo Server. The current per field cache control that Apollo exposes is quite powerful. It has a real impact on the response times of our queries.

Saying that, I still have a few REST data sources that are quite slow and every time my cached field value expires, I can notice how the REST endpoint is being hit and the cache recreated. This is of course, expected behaviour.

I was wondering if it would be possible to have a @cacheControl directive hint that would instruct the cache to default to returning cached data even when it’s stale and in that specific case, to update the key’s value on the background.

This would mean that only the first uncached request would have to wait for the rest endpoint to respond.

From the look of it, it seems that this specific use case would be impossible to implement without both a custom KeyValueCache and a custom apollo-server-plugin-response-cache implementation.

I can see two modifications being required, the first one on the Interface exported by KeyValueCache, having a method in the following form would greatly simplify implementing revalidation on the background. This would give the cache the responsibility of creating the cached value when and if required.

const value = await cache.getOrSetOnStale('key', async () => {
  const updatedValue = await generateValue();
  await cache.set('key', updatedValue);
  return updatedValue;
});

The second change would require having apollo-server-plugin-response-cache pass in the hint that the value should be revalidated in the background if it’s stale.

Reference http stale while revalidate implementation

For some reference, see this article: https://www.fastly.com/blog/stale-while-revalidate-stale-if-error-available-today

[stewartmcgown]

Yep! Which is why I combined it with response-cache-plugin so it would work with existing clients using the POST /graphql verbage. When switching to automatic persisted questions over GET for clients automatic CDN caching works great, but I wanted to use it with a response cache plugin.

[matchu]

(Ah yeah, you probably already understand this but I should say it out loud for clarity: my gist only implements the HTTP Cache-Control header, and it relies on a separate cache layer like Varnish, Cloudflare, Fastly, or Vercel’s built-in cache, to actually perform the response caching according to what the Cache-Control header says!)