Issues with binary files

Hi,

I really wanted to use r2 as a modern alternative to request, but I ran into issues with binary files. When I try to download for example our company logo, it all works fine:

> (async function () { console.log(await require('r2').get('https://wizcorp.jp/images/logo.png').arrayBuffer); console.log('done'); })();

output:

Promise {
  <pending>,
  domain: 
   Domain {
     domain: null,
     _events: 
      { removeListener: [Function: updateExceptionCapture],
        newListener: [Function: updateExceptionCapture],
        error: [Function: debugDomainError] },
     _eventsCount: 3,
     _maxListeners: undefined,
     members: [] } }
> ArrayBuffer { byteLength: 3874 }
done

But on a larger file (not sure if “large” has anything to do with it though), it just sort of hangs there:

(async function () { console.log(await require('r2').get('http://ipv4.download.thinkbroadband.com/5MB.zip').arrayBuffer); console.log('done'); })();

output:

Promise {
  <pending>,
  domain: 
   Domain {
     domain: null,
     _events: 
      { removeListener: [Function: updateExceptionCapture],
        newListener: [Function: updateExceptionCapture],
        error: [Function: debugDomainError] },
     _eventsCount: 3,
     _maxListeners: undefined,
     members: [] } }
> 

Replacing .arrayBuffer with .response does seem to return a response object of some kind, but I don’t know how to extract data from that:

> (async function () { console.log(await require('r2').get('http://ipv4.download.thinkbroadband.com/5MB.zip').response); console.log('done'); })();

output:

Promise {
  <pending>,
  domain: 
   Domain {
     domain: null,
     _events: 
      { removeListener: [Function: updateExceptionCapture],
        newListener: [Function: updateExceptionCapture],
        error: [Function: debugDomainError] },
     _eventsCount: 3,
     _maxListeners: undefined,
     members: [] } }
> Response {
  size: 0,
  timeout: 0,
  [Symbol(Body internals)]: 
   { body: 
      PassThrough {
        _readableState: [ReadableState],
        readable: true,
        domain: [Domain],
        _events: [Object],
        _eventsCount: 7,
        _maxListeners: undefined,
        _writableState: [WritableState],
        writable: true,
        allowHalfOpen: true,
        _transformState: [Object] },
     disturbed: false,
     error: null },
  [Symbol(Response internals)]: 
   { url: 'http://ipv4.download.thinkbroadband.com/5MB.zip',
     status: 200,
     statusText: 'OK',
     headers: Headers { [Symbol(map)]: [Object] } } }

[jacktuck]

Ah so you just want buffer?

require('r2').get('http://ipv4.download.thinkbroadband.com/5MB.zip').response.then(res => res.buffer()).then(console.log)

Promise {
  <pending>,
  domain:
   Domain {
     domain: null,
     _events: { error: [Function: debugDomainError] },
     _eventsCount: 1,
     _maxListeners: undefined,
     members: [] } }
> <Buffer 79 99 f5 09 c5 c2 c8 4e 36 f7 44 2e ce ee 79 2d 14 49 55 cd 50 66 dd 85 8c 56 d3 2e eb 28 d8 55 4c 46 7c 4f 97 a7 2d a8 21 b4 26 9e e8 51 e5 36 40 25 ... >

that work for you?

[DevBrent]

@ronkorving This is related to the use of .clone() in the wrapper r2 has for .text .json .arraybuffer etc. You can await (await r2(url).response).buffer()

The writestream is paused because the original response object’s readstream never reads any data when we clone in r2. It’s a node-fetch/streams problem but others in the community are working around it by not using clone.

See: https://github.com/mikeal/r2/issues/48#issuecomment-415166175