Add a pool size to QueryCursor#eachAsync

I wrote a little wrapper that pretty much acts like QueryCursor#eachAsync(fn, [callback]) but with a small difference which basically allows it to have a pool of items available, enabling to exec faster and to make stuff in parallel.

Use case:

Each fn given to eachAsync is a promise that makes some IO and takes 5s to run. eachAsync would wait it to be finished before executing a new one but it doesn’t mean you cannot execute multiple at once and to always have a pool of X fn executing at the same time, which would enable a higher throughput.

Hence, if eachAsync could buffer the next Y items to not have to wait for a new IO between mongo and the Node app it would enable a higher throughput too.

API suggestion:

QueryCursor#eachAsync(fn, [options], [callback]) where options is something like:

{
    poolSize: >=1, // How many [fn] will be executed in parallel
    bufferSize: >=0 // How many items shall be retrieved in advance
}

If it has any interest I could think about making a PR for it.

Thanks for the good job guys!

[YouriT]

What about:

{
    simultaneous: >=1, // How many [fn] will be executed in parallel
    bufferSize: >=0 // How many items shall be retrieved in advance
}

You like both enhancements?

Will work on something in the upcoming days I believe.

[vkarpov15]

Sure. In the meantime id recommend you just use co to implement this in your app, it should be a while loop with maybe 10 lines of code