Question: memory usage

I use the following code to generate a thumbnail image on my parse sever. It does

• fetch the image from an URL to a buffer
• run sharp on it to resize, save result in buffer
• create parse file from buffer
• save the parse file to DB

Parse.Cloud.httpRequest({url: coverartUrl})
.then(function(response) {
	var sharp = require("sharp");
	return sharp(response.buffer).resize(size, size).toFormat('jpeg').toBuffer()
}).then(function(buffer) {
	console.log("Buffer returned, creating Parse File");
	var base64Buffer = buffer.toString("base64");
	var filename = channel.get("coverart").name() + "_thumbnail.jpg";
	var file = new Parse.File(filename, { base64: base64Buffer }, "image/jpeg");
	return file.save();
}).then(function(file) {
	console.log("Thumbnail saved");
	return channel.set(coverartAttribute, file);
});

I run this code on about 30 images. This will cause nodejs to use > 500MB memory afterwards. Garbage collection does not clean the memory even after waiting some time.

I tried to fix it by explicitly setting some buffers to null. But it did not help. Any idea what I am doing wrong and what is causing the memory leak?

[JesusIslam]

@papandreou

I am using v7.8.0 atm. Turned out the culprit was initializing GCS module inside the async function that is called inside a loop. I solved it by initializing only once outside of the loop. After I do that the memory leak problem got away. This makes me unable to ran it using dynamic bucket to upload the picture to, but it works for my current use case, so yea.

Not sharp’s fault I think.

[papandreou]

@JesusIslam, have you tried upgrading to node.js 7? I’ve found that V8 5.4+ is more eager to return memory to the OS, which made a big difference in a long-running app that uses sharp’s stream API in a way that’s similar to what you’re doing.

Also, if you’re on linux, try adding you app to a cgroup and specify a memory.soft_limit_in_bytes value for it.