bulk queue with timer - how to construct? cargo? timeout?
See original GitHub issueI am trying to figure out how to use async
to create a “cargo-timeout-queue”.
Essentially, I am trying to cache and throttle downstream i/o behaviour. Think of it as “cargo with minimum batch size and timeout to override.”
const q = async.cargo_min_with_timeout(handler, 10, 5000);
The above would:
- create a cargo-like queue
- that would not call
handler
(first arg) until it has 10 (second arg) items to send together - but if it is at least 5000 (last arg) milliseconds since last time a handler was called and there are items on the queue, call
handler
with what we have
This allows me to combine input objects for efficient downstream i/o processing by handler
, but ensures that it doesn’t wait too long. There is a tension between batch size - larger can be better - and delay - I don’t want to wait too long.
Is there some way to combine async
functions to get this behaviour? Or do I need to build it?
Thanks. Once again, async
is a mainstay in my JS work. 😃
Issue Analytics
- State:
- Created 6 years ago
- Comments:12 (1 by maintainers)
Top Results From Across the Web
async-timed-cargo - npm
Start using async-timed-cargo in your project by running `npm i ... is better for message batch process as it waits until the queue...
Read more >Container Timeout - Codehaus Cargo
Cargo has a timeout for container start and stop operations. If the time taken to start/stop a container exceeds the timeout period the...
Read more >Configuring the storage of events for timeout nodes - IBM
You can use a Timer policy to control the storage of events for TimeoutNotification and TimeoutControl nodes.
Read more >46.6 Receiving Messages Asynchronously Using a Message ...
The application client sends messages to the queue, which was created administratively using the Administration Console. The JMS provider (in this case, ...
Read more >package.rs.html -- source - Docs.rs
Source to the Rust file `/home/cratesfyi/.cargo/registry/src/github.com-1ecc6299db9ec823/cargo-semverver-0.34.0/src/cargo/core/package.rs`.
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Also, we are missing a
cargoQueue
function – where you can set both thepayload
size andconcurrency
. I believe it is already implemented internally, just not exposed publicly.Closing in favor of #1555