Way to determine unprocessed items from async.map() after error is emitted

Hi,

Whilst using map(…) is there a way when handling an error emitted from the iteratee function to determine how many items have not yet been processed from the collection supplied?

I am using map to perform multiple outgoing HTTP requests (using the node request library) for an array of different urls\params, etc. Part way through making any of these requests I may get a particular error from the target server which I can handle, but I then want to re-process the current item being worked on and then any remaining ones that map has not yet picked up.

I did wonder about maybe setting a flag on each item in my collection that has been worked on successfully (without error), and then when the error gets emitted that I am interested in, handle it accordingly. Then perhaps create a new array from the items with the flag set to false for those not yet processed and perform a further map over these, making sure I invoke the original final callback from the original map.

Not sure if this makes any sense, but is there any way to achieve what I have described above?

[hargasinski]

Hi @parky128, thanks for the question!

Would wrapping the iteratee with reflect work? reflect always passes a result object to the callback, so even if one of the iteratee functions errors, map would finish. Then you could just iterate over the results object from map, check which ones have an error property, and then handle it accordingly. You wouldn’t have to reprocess any items that map may not have picked up.

async.map(coll, async.reflect(function(val, callback) {
  // your request code
}, function(err, results) {
  // err will always be null now
  results.forEach(function(result, i) {
    if (result.error) {
      // your code for handling errors
      // if `coll` is an array, you could access the value that caused 
      // this error through `coll[i]` as `map` preserves the order for arrays 
    } else {
      // otherwise `result.value` will contain the result of that `iteratee` call
    }
  });
});

Otherwise, to answer your question, map always returns an array. You could iterate over that array and check which values are undefined. Those will correspond to item(s) that either errored, passed undefined to their callback, were in progress when the error occurred or had not started. The reflect approach is probably the safer option though, as undefined may be a valid result from an iteratee call.

[parky128]

@hargasinski - I have ended up using the async.reflect approach and this is working nicely for me, gives me full visibility of all items that gave an error 👍

Thanks!