loadData problem with ReadableStream

Slightly odd bug to report that might be me simply misusing this (great!) library. In nodejs 0.10.33 everything works great, but in the latest nodejs (0.12.4) I am seeing an error when using jBinary.loadData in conjunction with a ReadableStream

buffer.js:201
  length += list[i].length;
                   ^
TypeError: Cannot read property 'length' of null
    at Function.Buffer.concat (buffer.js:201:24)
    at PassThrough.<anonymous> (.....\node_modules\jbinary\dist\node\jbinary.js:451:39)
    at PassThrough.emit (events.js:129:20)
    at _stream_readable.js:908:16
    at process._tickDomainCallback (node.js:381:11)

I’m using nodejs v0.12.4 and the ‘async’ library. Here is the offending code:

async.waterfall([
     var stream = require('fs').createReadStream('path/to/file');
     next(null, stream);
},
function parseFile(stream, next) {
  jBinary.loadData(stream, MY_TYPESET_HERE, function(err, binary) {
      // execution does not get this far
      var myData = jbinary.readAll();
  });
},

jBinary.loadData basically blows up when I pass a readableStream to it. I’m very new to nodejs so I’m not sure what the issue is, but presumably nodejs’ ReadableStream has changed from node 0.10.x -> 0.12.x and maybe jBinary does not officially support ReadableStream as a data source? Is this even an issue in jBinary to start with?

This code fragment is actually part of a much larger project that uses AWS’s S3 storage API. I’m retrieving the file from an S3 bucket and not simply using a static file. I can work around this issue by either downloading the file to temp storage and then loading it that way (and avoiding use of ReadbleStream), or avoiding storing the S3 file to disk entirely, and keeping everything in-memory Buffer.

As an aside, what is jBinary’s preferred approach to data sources? When loading a file from disk does it load it all into memory anyway, or does it read the file chunk-by-chunk? If the file is always loaded into ram then I guess I can just use Buffers and be done with it

cheers & thanks for a really nice library

[RReverser]

Do you mean that the tests are failing?

No, the autorelease was setup via Travis - basically it published releases whenever a new version was tagged, but 1) that doesn’t work anymore and 2) local builds fail for some obscure reasons I don’t have time to debug either.

If you can get builds working again in a PR, I’m happy to publish a release manually.

[RReverser]

I’m still running into this same problem using node.js 14.16.1. 😢 Wasn’t this fixed by a PR from @strobejb six years ago?

Hm, I’d think so too. Although it looks like I might not have done an npm release after that…