Investigate using `diskcache` for persistent MAST query caching

We could look into using the diskcache package to cache MAST search operations:

https://pypi.org/project/diskcache/

The package is a disk-based key-value store which makes it easy to specify an expiration time for each entry. This would allow us to e.g. cache search queries for 24 hours, which would significantly speed up searches across different Python processes, whilst largely avoiding the problem of stale search data.

[orionlee]

@barentsen this item is to look into possibility of an (interim) MAST query caching until astroquery supports MAST query caching, right?

Thoughts:

• request-cache can be useful in astroquery layer but probably not helpful for us. More specifically, MAST queries use long polling such that it could involve multiple HTTP calls (i.e., multiple requests). It’s the reason why the current astroquery cache does not work for MAST calls (astroquery’s current cache is also at request level).
• diskcache seems to be promising . The @cache.memoize decorator works on the level comparable to memoization.

[barentsen]

@JWCook Thank you for reaching out. requests-cache looks like a fabulous package!

The vast majority our requests calls are handled by the astroquery package, so I wonder if we ought to look into looking requests caching there.