question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Node.js axios upload large file using FormData, read all data to the memory, cause out of memory issue.

See original GitHub issue

Describe the bug

Node.js axios upload large file using FormData, read all data to the memory, cause out of memory issue.

To Reproduce

  const formData = new FormData();
  formData.append('file', fs.createReadStream(path), { filepath: path, filename: basename(path) });
  formData.append('videoId', videoId);
  await axios.post(UPLOAD_MEDIA_URL, formData, {
    headers: {
      'Content-Type': `multipart/form-data; boundary=${formData.getBoundary()}`,
      'Authorization': `Bearer ${await token()}`,
    },
    maxBodyLength: Infinity,
    maxContentLength: Infinity,
  });

Expected behavior

Read file and upload by stream, without loading all data in the memory.

Environment

  • Axios Version [0.21.1]
  • Adapter [HTTP]
  • Node.js Version [v14.16.0]
  • OS: [Alpine in docker, OSX 10.15.6]
  • Additional Library Versions [form-data@4.0.0]

Additional context/Screenshots

Not needed.

Issue Analytics

  • State:open
  • Created 2 years ago
  • Reactions:7
  • Comments:10

github_iconTop GitHub Comments

4reactions
MengLi619commented, Jan 26, 2022

Just give a clue for it, after I add the maxRedirects: 0 option, the upload will use much less memory than before. maxRedirects: 0 will switch the transport from http/https to follow-redirects, it seems follow-redirects use much more memory when uploading big file. Hope anyone can optimize this problem, thanks.

0reactions
d11rcommented, Sep 11, 2022
Read more comments on GitHub >

github_iconTop Results From Across the Web

Unable to upload large files using nodejs/axios - Stack Overflow
I am using the following code to upload files, it works fine for files upto 2 gb, but beyond that it throws an...
Read more >
Send a File With Axios in Node.js - Maxim Orlov
In this article, you'll learn how to send files and associated data by constructing a form. We'll cover the two file types —...
Read more >
A complete guide to CSV files in Node.js - LogRocket Blog
In this article, we will learn how to manage CSV files in Node. ... in the above examples, Node will read the entire...
Read more >
Spring Boot File upload example with Multipart File - BezKoder
In this tutorial, I will show you how to upload and download files with a Spring Boot Rest APIs to/from a static folder....
Read more >
S3 direct file upload using presigned URL from React and Rails
const { url, file_key: fileKey, s3_upload_params: fields } = uploadParams; const formData = new FormData(); formData.append('Content-Type', ' ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found