Bulk Load / Loader that will use loadAll()
See original GitHub issueHey, is there any plan for releasing Cache implementation that would use Loader’s loadAll()? It’d help me a lot with my use case, where I fetch data from HTTP endpoint which supports requests by many IDs. So for 100k objects in cache I could do 1k HTTP requests if each request would fetch 100 IDs instead of 100k requests 1 by 1.
Expected behaviour:
- collect all cache entries eligible to refresh and load them via single
loadAll
method
A nice to have feature would be an option about the size of the batch OR time limit eg.: the batch size is 1.000, but after 2 minutes there’re only 512 keys eligible to load so the cache loads all the 512 keys and reset the timer
Issue Analytics
- State:
- Created 4 years ago
- Reactions:1
- Comments:36 (21 by maintainers)
Top Results From Across the Web
Bulk load with custom load implementation - Google Groups
loadAll (true). When we do this, Hazlecast calls loadAllKeys() on corresponding mapStore and distributes the keys among partition threads to load the subset....
Read more >CacheLoader (Guava: Google Core Libraries for Java 31.0-jre ...
Returns a cache loader that uses function to load keys, without supporting either reloading or bulk loading. This allows creating a cache loader...
Read more >Google Guava CacheBuilder - Issue With Bulk Load From ...
I'm trying to override Google Guava's CacheLoader loadAll() method. I'm wanting to do a bulk lookup and retrieve all the documents in the ......
Read more >CacheLoader.loadAll(Collection keys) used by Coherence 3.0 ...
We would like to make use of this mechanism in the CacheLoader to implement the bulk load request. The throughput of single requests...
Read more >DataStax Bulk Loader Pt. 1 — Introduction and Loading
In addition to the bulk load and bulk unload use cases, dsbulk aids in migrating data to a new schema and migrating data...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Just released a preview with bulk cache loader support of
getAll
,loadAll
andinvokeAll
. https://github.com/cache2k/cache2k/releases/tag/v2.1.1.AlphaThere will be more tests with concurrency and further optimizations.
@cybuch
I plan to remove the old loader classes in
org.cache2k.integration
and some other deprecated things to clean up the interface. Expect to be finished in two weeks since I am in vacation next week.