question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Bulk Load / Loader that will use loadAll()

See original GitHub issue

Hey, is there any plan for releasing Cache implementation that would use Loader’s loadAll()? It’d help me a lot with my use case, where I fetch data from HTTP endpoint which supports requests by many IDs. So for 100k objects in cache I could do 1k HTTP requests if each request would fetch 100 IDs instead of 100k requests 1 by 1.

Expected behaviour:

  • collect all cache entries eligible to refresh and load them via single loadAll method

A nice to have feature would be an option about the size of the batch OR time limit eg.: the batch size is 1.000, but after 2 minutes there’re only 512 keys eligible to load so the cache loads all the 512 keys and reset the timer

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Reactions:1
  • Comments:36 (21 by maintainers)

github_iconTop GitHub Comments

4reactions
cruftexcommented, Dec 24, 2020

Just released a preview with bulk cache loader support of getAll, loadAll and invokeAll. https://github.com/cache2k/cache2k/releases/tag/v2.1.1.Alpha

There will be more tests with concurrency and further optimizations.

3reactions
cruftexcommented, Jul 22, 2021

@cybuch

BTW do you know when you’ll be releasing this feature as regular version?

I plan to remove the old loader classes in org.cache2k.integration and some other deprecated things to clean up the interface. Expect to be finished in two weeks since I am in vacation next week.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Bulk load with custom load implementation - Google Groups
loadAll (true). When we do this, Hazlecast calls loadAllKeys() on corresponding mapStore and distributes the keys among partition threads to load the subset....
Read more >
CacheLoader (Guava: Google Core Libraries for Java 31.0-jre ...
Returns a cache loader that uses function to load keys, without supporting either reloading or bulk loading. This allows creating a cache loader...
Read more >
Google Guava CacheBuilder - Issue With Bulk Load From ...
I'm trying to override Google Guava's CacheLoader loadAll() method. I'm wanting to do a bulk lookup and retrieve all the documents in the ......
Read more >
CacheLoader.loadAll(Collection keys) used by Coherence 3.0 ...
We would like to make use of this mechanism in the CacheLoader to implement the bulk load request. The throughput of single requests...
Read more >
DataStax Bulk Loader Pt. 1 — Introduction and Loading
In addition to the bulk load and bulk unload use cases, dsbulk aids in migrating data to a new schema and migrating data...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found