question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Accessing a writable stream for `context.response`

See original GitHub issue

Firstly, thanks for your effort so far in creating oak!

I’m trying trying to implement a proxy that writes data from a readable stream to the response of an incoming request in a granular fashion using streams.

In express I would do something like this (if I wanted to manipulate the chunks in some way):

readableStream
    .on('data', chunk => {
        response.write(chunk);
    })
    .on('end', () => {
        response.status(statusCode);
        response.end();
    });

Or just:

readableStream.pipe(writableStream);

Unfortunately I’m stuck right now as I can’t find anyway to access an underlying WriteableStream interface for the response.

Is this possible?

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:8

github_iconTop GitHub Comments

3reactions
patrickkunkacommented, Jun 11, 2020

FYI for anyone else that needs to convert from a whatwg ReadableStreamDefaultReader to Deno.Reader while maintaining Uint8Array throughput, this is the util I ended up creating:

https://deno.land/x/bandwidth_throttle_stream@1.1.0/Util/readerToDenoReader.ts

0reactions
patrickkunkacommented, Jun 6, 2020

Thanks a lot both. I was a bit confused by the fact that the response body could take a Reader but not a Writer, but now I see this is fundamental to the design goals of Deno. Just required a bit of a shift in thinking.

The below is what I ended up with and its working nicely.

Firstly, my proxy uses fetch to proxy the incoming request to another server, and the goal is then that I can stream chunks from the fetch response directly back to the server response while manipulating throughput on the fly.

//  `body` provides access to a `ReadableStream` instance for the fetch request

const {body} = await fetch(url);

// That stream can then be transformed as necessary with `.pipeThrough()`,
// and finally turned into a `reader` using `.getReader()`:

const reader = body
    .pipeThrough<Uint8Array>(myCustomTransformStreamHere))
    .getReader();

// However, the `reader` is an instance of the standard `ReadableStreamDefaultReader`,
// and not `Deno.Reader`, so some final conversion is needed between the two:

const denoReader: Deno.Reader =  {
    read: async (p) => {
        const {value} = await reader.read();

        if (!value) return null;

        p.set(value);

        return value.length;
    }
};

context.response.body = denoReader;

The final conversion between reader types is just a crude MVP for now. There is an important edge case that needs to be covered where value.length is greater than p.length, and a queue would need to be implemented, but the basic approach is working nicely.

Thanks for your help.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Streams—The definitive guide - web.dev
The ReadableStream() constructor creates and returns a readable stream object from the given handlers. There are two types of underlying source:
Read more >
Writing a stream to the response in ASP.Net - Stack Overflow
1 Answer 1 ... Response.Write() will only write content, you have to set the headers before calling this. You could enumerate the headers...
Read more >
Using writable streams - Web APIs | MDN
To create a writable stream, we use the WritableStream() constructor; the syntax looks complex at first, but actually isn't too bad. The syntax ......
Read more >
Working with Node.js streams - LogRocket Blog
Writable streams : The writable stream is responsible for writing data in specific formats to files. Duplex streams: Duplex streams are streams ......
Read more >
Understanding Streams In NodeJS - Medium
We can use writable streams to write data to terminal, http response, or to a file. We use the write a method to...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found