question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Memory leak in http-proxy dependency when client closes connection prematurely

See original GitHub issue

Checks

Describe the bug (be clear and concise)

This is an issue of http-proxy, but I decided to report it here as well because:

  • http-proxy-middleware is affected by this issue
  • last http-proxy version was released in 2020, so I don’t expect them to fix it soon
  • this issue could be fixed on http-proxy-middleware side

Here is a complete description of this issue: https://github.com/http-party/node-http-proxy/issues/1586

Step-by-step reproduction instructions

Here is a minimal example to reproduce the above-mentioned memory leak:

const express = require('express');
const { createProxyMiddleware } = require('http-proxy-middleware');

const server = express()
	.use(createProxyMiddleware({ target: 'http://127.0.0.1:5000' }))
	.listen(7000);

server.once('listening', () => {
	console.log('server is listenning');
});

I used http-server to create an upstream server:

npx http-server . -p 5000

Memory leak appears when client sends a http request, but then closes connection prematurely after receiving HTTP headers without waiting for HTTP body.

I used cURL to simulate this behavior:

curl http://localhost:7000/15142e1388c685f57c58e0babbede1f1.jpg

Memory leak could be reproduces with cURL when requesting any binary file from a TTY terminal, since cURL doesn’t support printing binary data to terminals and just closes the connection.

Expected behavior (be clear and concise)

Memory should not leak

How is http-proxy-middleware used in your project?

↪ npm ls http-proxy-middleware
culture-ru-gateway@1.0.6-alpha.0 /home/leonid/Documents/culture/gateway
└── http-proxy-middleware@2.0.6

What http-proxy-middleware configuration are you using?

{
	retry: {
		factor: 2,
		randomize: true,
		retries: 30,
		minTimeout: 10,
		maxTimeout: 500
	},
	xfwd: true,
	logLevel: 'warn'
}

What OS/version and node/version are you seeing the problem?

System:
    OS: Linux 5.13 Ubuntu 20.04.4 LTS (Focal Fossa)
    CPU: (8) x64 Intel(R) Core(TM) i7-10510U CPU @ 1.80GHz
    Memory: 5.11 GB / 15.34 GB
    Container: Yes
    Shell: 5.0.17 - /bin/bash
  Binaries:
    Node: 16.15.0 - ~/.nvm/versions/node/v16.15.0/bin/node
    npm: 8.5.5 - ~/.nvm/versions/node/v16.15.0/bin/npm
  Managers:
    Apt: 2.0.8 - /usr/bin/apt
    Cargo: 1.49.0 - ~/.cargo/bin/cargo
    pip3: 20.0.2 - /usr/bin/pip3
    RubyGems: 3.1.2 - /usr/bin/gem
  Utilities:
    CMake: 3.16.3 - /usr/bin/cmake
    Make: 4.2.1 - /usr/bin/make
    GCC: 9.4.0 - /usr/bin/gcc
    Git: 2.25.1 - /usr/bin/git
    Ninja: 1.10.0 - /usr/bin/ninja
    FFmpeg: 4.2.4 - /usr/bin/ffmpeg
  Servers:
    Nginx: 1.18.0 - /usr/sbin/nginx
  Virtualization:
    Docker: 20.10.12 - /usr/bin/docker
  IDEs:
    Nano: 4.8 - /usr/bin/nano
    Sublime Text: 4126 - /usr/bin/subl
    VSCode: 1.67.1 - /usr/bin/code
  Languages:
    Bash: 5.0.17 - /usr/bin/bash
    Perl: 5.30.0 - /usr/bin/perl
    Python3: 3.8.10 - /usr/bin/python3
    Ruby: 2.7.0 - /usr/bin/ruby
    Rust: 1.49.0 - /home/leonid/.cargo/bin/rustc
  Databases:
    MongoDB: 4.4.14 - /usr/bin/mongo
  Browsers:
    Chrome: 101.0.4951.64

Additional context (optional)

Here is a code I used to fix http-proxy memory leak in my project:

proxy.on('proxyRes', (proxyRes, req, res) => {
	const cleanup = (err) => {
		// cleanup event listeners to allow clean garbage collection
		proxyRes.removeListener('error', cleanup);
		proxyRes.removeListener('close', cleanup);
		res.removeListener('error', cleanup);
		res.removeListener('close', cleanup);

		// destroy all source streams to propagate the caught event backward
		req.destroy(err);
		proxyRes.destroy(err);
	};

	proxyRes.once('error', cleanup);
	proxyRes.once('close', cleanup);
	res.once('error', cleanup);
	res.once('close', cleanup);
});

The actual code could be simpler since:

  • it should be enough to listen only for 'close' events since they are always emitted after 'error' events (unless emitClose is false)
  • it’s probably not necessary to call removeListener manually, because those listeners should not prevent streams from being garbage collected (but who knows, I certainly don’t want to test it)

Issue Analytics

  • State:open
  • Created a year ago
  • Reactions:4
  • Comments:8

github_iconTop GitHub Comments

1reaction
lbeschastnycommented, Sep 14, 2022

I just added a hotfix branch to my reproduction example repository to demonstrate that it fixes the memory leak in question

0reactions
lbeschastnycommented, Nov 10, 2022

Here is another fix for this memory leak: http-party/node-http-proxy#1559.

It was published to npm as http-proxy-node16.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Bug #1493303 “[OSSA 2016-004] Swift proxy memory leak on ...
It looks like the Swift proxy will leak memory if the connection is closed and the full response is not read. This opens...
Read more >
Changelog - curl
... cmake: link curl to its dependencies with PRIVATE · cmake: remove APPEND in export(TARGETS) ... schannel: fix memory leak due to failed...
Read more >
Nginx 1.14
*) Feature: the mail proxy now logs client port on connect. *) Bugfix: memory leak if the "ssl_stapling" directive was used. Thanks to...
Read more >
Spring Cloud Gateway memory leak with non-standart ...
Real connection from pool is not closed or what? Dependencies: spring boot 2.7.5; spring cloud gateway 3.1.4; netty-all 4.1.84Final ...
Read more >
ZooKeeper Release Notes
[ZOOKEEPER-1676] - C client zookeeper_interest returning ZOK on Connection ... Prevent resource leak in various classes; [ZOOKEEPER-2073] - Memory leak on ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found