multistream not consuming entire stream due to race condition. reproduction included.

I’m running into an issue where multistream doesn’t consume the entire stream. Not sure why. If you follow the instructions and run node multistream.js lots of times, you’ll see intermittent content length mismatches.

I included node passthrough.js to provide an example that never fails. multistream.js and passthrough.js are nearly identical, only difference being piping to multistream vs piping to a passthrough stream.

https://www.diffchecker.com/bi5wMRCW

Reproduction steps

$ git clone https://github.com/guanzo/multistream-debug
$ cd multistream-debug
$ npm i 

# Run 20+ times, no failures
$ node passthrough.js

# Run 20+ times, intermittent failures
$ node multistream.js

here’s what i’m seeing

Edit: I found this bug after seeing webtorrent make never ending requests due to an invalid torrentFile. create-torrent uses multistream, and whenever this bug occurs, an invalid torrentFile is returned. Webseed tries to verify a valid chunk against an invalid torrentFile’s chunk hash. The verification fails, so webtorrent retries endlessly.

Edit: $ node multistream.js works with multistream@3.0.0. This commit seems to be the problem.

https://github.com/feross/multistream/commit/0813df4cc0227b61ad24f314d4970b17d8b45941

Removing && this._drained fixes the bug. Thoughts?

[srolel]

I’m also seeing this, is the fix going to be published?

[johnsusek]

Can confirm #51 fixes this issue for me