question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Stream file does not seem to work.

See original GitHub issue

Describe the bug

Unable to return a fs.createReadStream from an endpoint. There are plenty of examples for fetching a file from URL, and sending as stream, but sometimes a log file or file located on machine needs to be sent.

Reproduction

When attempting to return a fs.createReadStream via endpoint like so:

readStream = await fs.createReadStream(file) 
                     return {
                         
                         body:readStream,
                         headers: responseHeaders
                         };

an error results:

Node streams are no longer supported — use a ReadableStream instead
Error: Node streams are no longer supported — use a ReadableStream instead

These issues below relate in they attempt to stream or talk about documentation for examples missing, but do not talk about an opened file and how to actually send the file via a stream, for example a giant 1GB file: https://github.com/sveltejs/kit/issues/5344 https://github.com/sveltejs/kit/issues/5412 https://stackoverflow.com/questions/72894078/sveltekit-node-streams-are-no-longer-supported

Logs

Postman call to endpoint results in error above.

System Info

Chrome
Node v18.2.0
		"@sveltejs/adapter-auto": "next",
		"@sveltejs/kit": "next",
		"autoprefixer": "^10.4.7",
		"eslint": "^8.16.0",
		"eslint-config-prettier": "^8.3.0",
		"eslint-plugin-svelte3": "^4.0.0",
		"postcss": "^8.4.14",
		"prettier": "^2.6.2",
		"prettier-plugin-svelte": "^2.7.0",
		"svelte": "^3.44.0",
		"svelte-preprocess": "^4.10.7",
		"tailwindcss": "^3.1.6",
		"vite": "^3.0.0"
	},

Severity

annoyance

Additional Information

Maybe there is an alternative way to do this. But I don’t see any examples how to natively pipe this to a response in Sveltekit like you would in Express I returned a readableStream but the response is essentially the stream and not the contents image

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:5 (2 by maintainers)

github_iconTop GitHub Comments

1reaction
half-metalcommented, Jul 16, 2022

Yes! @repsac-by that works, thanks for showing how to do the return with Readable.toWeb Looking through the web stream standard @Conduitry, I’m not sure how anyone is to figure out that toWeb would ever solve anything Screen Shot 2022-07-16 at 11 45 47 AM

Many parts of the Node documentation on streams is confusing. Would be nice if they did a section on “modern” way to use streams and for the various common use cases. Their examples are limited, and a bit all over the place.

@AliBasicCoder yes that looks like it would work too, and that is what I was doing, except I was confused how you open a file that is stored locally, and then send that without more code. I’m still not sure where on the readableToReadStream you pull in that fs opened file, but I’ll play around with that eventually. For now, the solution from @repsac-by will work for what I need.

1reaction
AliBasicCodercommented, Jul 16, 2022

you just have to use web stream not node’s

here is code that will help you (remove type antotions if you aren’t using TS)

// converts web stream to node
export function readStreamToReadable(readStream: ReadableStream) {
  const result = new Readable({ read() {} });
  const reader = readStream.getReader();
  function push() {
    reader.read().then(({ done, value }) => {
      if (done) {
        return result.push(null);
      }
      result.push(value);
      push();
    });
  }
  push();
  return result;
}

// converts node stream to web
export function readableToReadStream(
  readable: Readable
): ReadableStream<Uint8Array> {
  return new ReadableStream({
    start(controller) {
      readable.addListener("data", (chunk: Buffer) => {
        controller.enqueue(new Uint8Array(chunk));
      });
      readable.addListener("end", () => {
        controller.close();
      });
      readable.addListener("error", (err) => {
        controller.error(err);
      });
    },
  });
}
Read more comments on GitHub >

github_iconTop Results From Across the Web

Google Drive File Stream Not Working – What to Do?
Methods to Fix “Google Drive File Stream Not Working” Problem · Try Disabling Firewall and Anti-virus Programs: · Try Checking and Modifying ...
Read more >
stream.read() returns 0 - MSDN - Microsoft
I have a WCF service set up mostly for uploading and downloading files. It is configured to use streaming and MTOM.
Read more >
Writing a stream into a file doesn't work propely - Stack Overflow
Try netStream.Flush();. I suspect what's happening is the stream is getting closed before the writing finishes. Sometimes the output stream ...
Read more >
Troubleshoot your YouTube live stream - Google Support
If you're having issues with your YouTube live stream, use the troubleshooting tips below. In addition to these tips, you can always post...
Read more >
Basics of I/O Streams and File I/O
File streams are like cin and cout, except that the flow is to and from a file. ... Note that this won't work...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found