question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

High memory usage when uploading large file.

See original GitHub issue

Env: Nodejs v8.9.1, googleapis v28.1.0, Windows 10. I tried to upload a large files around 2GB, I notice that the process is up to 3GB RAM. My code upload that file to Drive successful on 8GB ram system, failed on 1GB ram system.

      var fileMetadata = {
         "name": fileName
      }

      var media = {
          mimeType: mineType,
          body: fs.createReadStream(path)
      }

      self.drive.files.insert({
          resource: fileMetadata,
          media: media,
          fields: "id"
      }, function (err, file) {
        if (err) {
          logger.error("Error: " + err.message)
        }
        logger.info("Finished, id: " + file.data.id)
      })

8GB RAM untitled 1GB RAM image

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Reactions:2
  • Comments:9 (1 by maintainers)

github_iconTop GitHub Comments

2reactions
JustinBeckwithcommented, Jan 21, 2019

Pretty sure we have this fixed on master, and it’s going to be in v37 😃

2reactions
ghjbnmcommented, Aug 31, 2018

Did you try using maxRedirects: 0, works fine for me. Should look somewhat like this:

google.youtube('v3')["videos"]["insert"](
	{
		auth: oauth2Client,
		part: 'id,snippet,status',
		requestBody:requestBody,
		media:
		{
			body: readStream
		}
	},
	{
		maxContentLength: 128 * 1024 * 1024 * 1024,
		maxRedirects: 0
	});
Read more comments on GitHub >

github_iconTop Results From Across the Web

[SOLVED] Uploading a somewhat large file eats all the RAM
Uploading a somewhat large file eats all the RAM. Steps to reproduce. On Mattermost 4.4.3, upload a file of ~ 100MB, and see...
Read more >
How to limit memory usage when uploading large files
We are encountering heavy memory usage when uploading large files, leading to an out of memory exception. The memory usage seems to occur...
Read more >
High memory usage when uploading and manipulating many ...
I can open about:memory, click "Minimize memory usage" to trigger a garbage collection, press on "Measure" and the file data shows up under...
Read more >
RAM being maxed out while uploading huge file
I'm currently on the verge from buying 16GB of RAM. Hoping that can solve my issue.
Read more >
Apache2 memory usage when uploading large files
The problem is that when users upload a file, size of the apache process swells by almost the same amount. So if somebody...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found