Memory Leak

Describe the bug

I have a huge json file which I read using streams. I read this json file using JSONStream module. I do it like so

When JSONStream emits a json object axios posts this object to a webservice. There is no infinite loop anywhere, but axios is not releasing memory after finishing the request. In the image below you can see that after few minutes running, my node app reaches 12gb of memory! What is the problem?

To Reproduce

You can use the following code snippet to verify this issue

const fs = require("fs");
const path = require("path");
const JSONStream = require('JSONStream');
const axios = require('axios');
const axiosRetry = require('axios-retry');
const axiosRateLimit = require('axios-rate-limit');
const es = require('event-stream');
const winston = require('winston');

const logger = winston.createLogger({
	format: winston.format.combine(
		winston.format.timestamp({
            format: 'YYYY-MM-DD HH:mm:ss',
        }),
		winston.format.printf((info) =>
            JSON.stringify({
                timestamp: info.timestamp,
                message: info.message
            })
        )
    ),
    transports: [
		new winston.transports.File({filename: path.resolve(__dirname, 'errors.log'), level: 'error'}),
		new winston.transports.File({filename: path.resolve(__dirname, 'infos.log'), level:'info'})
    ]
});


axiosRetry(axios, { retries: 3, retryDelay: axiosRetry.exponentialDelay });
const http = axiosRateLimit(axios, { maxRequests: 20, perMilliseconds: 200 })

const postEvent = (uri, event, apiKey) => {
    return http.post(uri, event, {
        headers: {
            ...(apiKey && { 'api-key': apiKey }),
            'Content-Type': 'application/json'
        },
		maxRedirects: 0
    });
};

async function main(){

	const getStream = () => {
		const stream = fs.createReadStream(path.resolve('.', 'events.json'), { encoding: "utf8" });
		const parser = JSONStream.parse("*");
		return stream.pipe(parser);
	};

	getStream()
	.pipe(es.mapSync(async function (data){
		logger.info(data._id.$oid);
		try{
			await postEvent(data);
		}catch(e){
			logger.error(e)
		}
	}))
}

main();

Expected behavior

Memory should be released after request promise has been fullfiled.

Environment

• Axios Version 0.27.2
• Node.js Version 16
• OS: Ubuntu 20
• JSONStream: 1.3.5

[einhaus]

FYI I had this same issue, and the only solution was to remove axios-retry. It seems it was hanging on to the response for every request regardless of whether the request failed or not.

[DigitalBrainJS]

https://nodejs.org/api/stream.html#streampipelinestreams-callback