Relative redirect URLs target proxy server instead of upstream server

Hi! 👋

Firstly, thanks for your work on this project! 🙂

Today I used patch-package to patch follow-redirects@1.14.5 for the project I’m working on.

The problem I was running into is when using axios with a proxy and the upstream server returns a relative redirect URL, follow-redirects crafts an incorrect redirectUrl based on its cached this._currentUrl. It turns out that this URL is computed by using url.format(this._options), which doesn’t work correctly when a proxy server is configured because this._options is assumed to be an object containing valid Url parts, when it’s in fact an object containing ClientRequest options with no guarantee that their semantics matches Url parts.

More specifically, the axios http adapter that uses follow-redirects modifies the original options by setting the host, hostname and port to the proxy’s own host/port values, and path property to the full upstream server URL. (This, from what I know, is the correct way to configure a Node.js http(s) request to go through a proxy.)

As a result, parsing the ClientRequest options as if it were a URL results in a weird URL like http://<proxy-host-and-port>/https://<upstream-host-and-port>/<upstream-path>, and when follow-redirects handles a relative redirect URL, it then uses this incorrect currentUrl as the base URL to use with url.resolve, and causes the redirected URL to be based the proxy host rather than the upstream host. A proxy like mitmproxy then detects the redirect as an attempt to connect to itself through itself and immediately breaks the connection with a 502 error.

My solution is to try to resolve the request options’ path property against this (semantically risky) “current URL”. If path is an absolute URL, it will be used instead; if it’s relative, then the original “current URL” will be unchanged. This makes follow-redirects “proxy-friendly” without explicitly adding proxy features.

Here is the diff that solved my problem:

diff --git a/node_modules/follow-redirects/index.js b/node_modules/follow-redirects/index.js
index 7605d7d..eb0009c 100644
--- a/node_modules/follow-redirects/index.js
+++ b/node_modules/follow-redirects/index.js
@@ -277,7 +277,7 @@ RedirectableRequest.prototype._performRequest = function () {
   // Create the native request
   var request = this._currentRequest =
         nativeProtocol.request(this._options, this._onNativeResponse);
-  this._currentUrl = url.format(this._options);
+  this._currentUrl = url.resolve(url.format(this._options), this._options.path)
 
   // Set up event handlers
   request._redirectable = this;

Note that this was previously reported (https://github.com/follow-redirects/follow-redirects/issues/85) but the example snippet seemed to be incorrectly setup. In my case, I could fully reproduce this issue using mitmproxy and a local Node http server (very similar to that issue’s http server, actually). This setup worked flawlessly with curl, and it works with absolute redirects.

This issue body was partially generated by patch-package.

[RubenVerborgh]

Or really just https://www.twilio.com/blog/node-js-proxy-server since we’re using express already anyway in the tests.

[RubenVerborgh]

Firstly, thanks for your work on this project! 🙂

Well thank you for such a detailed problem description!

BTW, I am willing to draft up a proper PR with tests, if you are willing to consider taking this fix.

You’re amazing.

The first thing I’d need is one/a couple of example cases that fail. So those tests indeed, perhaps even with an actual proxy in the test case so we know it works properly.

I’m not 100% convinced yet by the fix; it might need to be a bit more elaborate, with explicit detection of the proxy case. But we can figure that out when the tests are in place.

Thanks in advance!