Client side caching may be overriding cache invalidation

I haven’t been able to fully research this as it’s quite complex and I’m not an expert on caching headers, but plan to in the future and want to write it down to make sure I’m not on a completely wrong path.

What I’m experiencing is that browsers that are accessing pages that are cached and invalidated using django-fancy-cache are still pulling a resource (from disk cache) on chrome. Hard refresh of the browser solves the issue which makes me believe this is an issue with browser caching results to disk.

Question - should items cached and invalided by fancy-cache include a no-cache; no-store header? Otherwise is there a good way to make sure that the browser doesn’t pull a stale response from its local cache?

I’ve looked into Etag and I suspect that might be a good alternative; as far as I can tell this would allow cache invalidation based on the hashed value of the response.

[YPCrumble]

@peterbe makes sense - and this is exactly what I’m currently doing everywhere I’m using fancy cache’s cache_page decorator.

What I was considering was whether A/ many users would want this functionality built-in with a setting, which makes sure they don’t forget to add the never_cache decorator, or B/ whether it would be possible to use an etag function that would further improve response times from the API, in that the cached response wouldn’t need to be sent if the entry in the fancy_urls dict were found with the same etag value.

This seems like a small optimization though so I’m going to close this.

[YPCrumble]

@peterbe reopening because I just realized that the latest version of our middleware, which follows the Django cache middleware, does not cache the response if the Cache-Control: private header is set.

the @never_cache decorator began adding this header in Django 3.2, so simply using the @never_cache decorator out of the box doesn’t work. For now I’m just going to make my own version of the header to remove the private header, but I could still see a use case for adding a setting like FANCY_ADD_NEVER_CACHE_HEADERS that would skip this check, like so:

if settings.FANCY_ADD_NEVER_CACHE_HEADERS is False:
    # Don't cache a response with 'Cache-Control: private'
    if "private" in response.get("Cache-Control", ()):
        return response

…and this would also add the never_cache headers elsewhere in the middleware so I don’t need to make my special version of never_cache.

A second alternative is I could add my modified version of @never_cache to this codebase as @fancy_never_cache and document what I’m doing. EDIT: Actually I would prefer to keep the private header as done in Django so this doesn’t really make sense.

None of this is absolutely necessary but thought to put this here in case anyone else encounters the issue in the future.