Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Stream behavior with errors

See original GitHub issue

If the call to the download function below fails with an error, stopOnError will cause the Promise to reject. Since this is doing parallel(10), will this code wait for the other 9 download operations to finish, or will it terminate them right away when it encounters the first error?

How will the behavior be different if I use mergeWithLimit(10) instead of parallel(10)?

  await new Promise((res, rej) => highland(range)
    .map(page => highland(download(page)))

Issue Analytics

  • State:open
  • Created 7 years ago
  • Comments:5

github_iconTop GitHub Comments

vqvucommented, Jan 23, 2017

Is it fine to have errors and stopOnError in the same stream?


Or as you said, done will throw if it handles an unhandled error, so maybe stopOnError(rej) isn’t necessary? But then how would I reject the Promise?

You can call rej inside errors instead of pushing the error along. Like this

    .errors((err, push) => {
      if (err instanceof MyError) {
      } else {
        rej(err) // Reject the promise.
        push(null, highland.nil) // stopOnError.

It’s important to call rej(err) first, before push nil. Pushing nil first may cause the stream to end and done(res) to execute before you get a chance to reject the promise, causing you to end up with a resolved promise instead of a rejected one. This is a quirk of the current implementation.

That said, I personally think it’s too easy to accidentally transpose the two lines, and it’s not immediately obvious that order matters, so I wouldn’t recommend doing this. Your solution using stopOnError(rej) is less error-prone.

is it fine to have an errors function without any push(null, myData)? In my example after doing log(err) would I need to do any kind of push? (The errors would already be removed from the stream and logged so I wouldn’t want to push any values for them back onto the stream.)

It’s perfectly fine to not push anything. That’s one of the primary uses of errors—to handle certain errors and filter them out from the stream.

vqvucommented, Jan 20, 2017

parallel will hold back errors the same way that it holds back stream values due to its ordering guarantee. mergeWithLimit will emit the error as soon as it gets it. stopOnError will stop the stream from consuming new data (and thus stop new operations from being started), but it does not cancel download operations that have already started.

Let’s say you have 15 download operations to process, your parallelism factor is 10, and the 3rd download fails.

With parallel

parallel will emit the error in order, so when your promise is rejected,

  • The first two download operations are guaranteed to have completed.
  • The 3rd download will have failed.
  • Up to the next 10 downloads (downloads 4-13) will have been started. “Up to” because the actual number depends on how fast the other downloads complete. These downloads will not be stopped, and you know nothing about them. In particular, any number of these downloads may have already completed.

With mergeWithLimit

mergeWithLimit will emit the error as soon as possible, so when your promise is rejected,

  • The 3rd download will have failed.
  • You know nothing about the other downloads. Depending on how fast the 3rd download executed, the actual number of started and completed downloads can be anything from 0 to 14.
  • The number of started but not yet completed downloads are guaranteed to be less than or equal to 10.

Using merge doesn’t tell you much about the download operations on errors. That’s the price you pay for the better memory usage and more parallelism. That said, if you want to know which downloads completed before the error, you can resolve the Promise that download returns with some id (say the page number) when the download completes. These values will be emitted before the error if the download completed before the error.

Note that you do not have to use stopOnError. There is also errors. It’s just there as an example, because you want to handle errors somehow (done will throw if it encounters an unhandled error). But it’s up to you how you want to handle them.

Read more comments on GitHub >

github_iconTop Results From Across the Web

How do I throw an error on a behaviour subject and continue ...
Retrieval/creation of the data: A stream, that will run once then then completes and/or throws an error whenever one occurs. When the data...
Read more >
Error Streams Sample
Error ports and error streams allow application authors to handle errors as data. You can apply local logic to error processing using StreamBase...
Read more >
Node.js Streams Inherit Error-Handling Behavior From ...
Ben Nadel demonstrates that the error handling behavior in Node.js streams is inherited from the EventEmitter module.
Read more >
Controlling Error Reporting Behavior and Intercepting Errors
If you don't use either of those, the error is written to the Error stream, and the current script block continues with the...
Read more >
Analysis of Behavioral Streams 1 Running head
observed streams of behavior; such scores then serve subsequent analyses. ... It places some burden on observers, however, and can be error prone...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found