Node.js support

Do you have anything in your roadmap regarding Node.js support?

The purpose is to allow other cross-env libraries to use fetch as the client.

From what I see now in master there are couple of things to be done:

1. Use XMLHttpRequest polyfill (for example https://github.com/pwnall/node-xhr2)
2. Wrap the code in something like UMD to make sure CommonJS and AMD compatibility
3. Export all classes and fetch() function instead of always writing to global
4. Add optional polyfill() method just like in es6-promise for AMD users
5. Take over NPM package name fetch, current one is unmaintained for 2 years

Is it worth investigation and making a PR? Does not seem to be too much effort.

[dgraham]

We receive a couple common requests, so I think it’s useful to summarize why we reject them. I spend lot of time considering these issues, and I don’t feel like I’m any closer to understanding why they recur. Here’s my perspective.

Background

Before looking at the requests, let’s cover some background. A polyfill is, by definition, a piece of JavaScript code that mimics a native web browser API that is still in development. It fills in missing functionality until the point where the browser supports the API natively, and then the polyfill is removed.

Browser APIs are not modules—they are all attached to the window global object. There is no support for optionally loading them, or loading some other version of them. They are native functionality that is built-in to the browser.

A polyfill is a somewhat unique piece of code, in the sense that it is intended to not be used at all in most browsers. A majority of visitors do not run this fetch polyfill. Their browser natively supports both window.fetch and the window.Promise global on which fetch depends.

Polyfills are valuable not because of their reach, but because they allow us to use new browser features before all major browsers have shipped them natively. This is code that is intended to be deleted within 2-3 years, once all browsers have built-in support.

Ok, so let’s discuss the two recurring requests.

1. Node.js support

The dream of Write Once Run Anywhere is being repeated under the banner of “isomorphic” JavaScript. The goal is to support both web browser runtime environments and Node.js server environments with a single piece of code. This is admirable, but having lived this dream once before with Java, it is not a reasonable long-term strategy. It does not work.

JavaScript must be optimized for its runtime: either a browser or a server process. Attempting to support both runtimes makes each of them worse. Some concrete examples follow.

Complexity

Fetch is a very well designed modeling of headers, requests, and responses, using asynchronous promises. This polyfill treats that design as a beautiful layer above an XMLHttpRequest implementation. We use XMLHttpRequest so you don’t have to!

But a Node.js server does not have XMLHttpRequest—it doesn’t make sense in a server environment. Node is not constrained by a standards process or several competing implementations. Anyone is free to write an HTTP library for Node.js, and subsequently, there are many great HTTP packages.

In order for fetch to support both a browser and a server we must introduce a plugin system where the HTTP subsystem may be swapped out for something else. We must then test the plugin system on all major browsers and then again with all popular HTTP libraries inside a Node.js server. This is complexity that we just don’t have time to support.

Lowest common denominator features

When designing for both a browser and a server, features that exist in only one runtime must not be used, and then we end up with something that works poorly in both environments.

The bitinn/node-fetch package is a great implementation of a fetch-like API optimized for Node.js. Browsing through the code reveals many non-trivial code changes required to optimize for servers:

1. URL protocol restrictions not present in a browser. A browser can fetch relative URLs because the request occurs within the context of a web page. Servers don’t have that context, so a restriction is needed to allow only absolute URLs.
2. Request compression, timeouts, custom user-agent, redirects. We would love to have timeout support in the browser, but it isn’t available. Because this is optimized for Node, we get to use timeouts on the server.
3. Headers access to internal data members. This would never be added to a browser implementation because it’s out of spec.
4. Chunked response body streaming. This is terrific and something that the fetch browser polyfill is unlikely to ever support because of limitations with XMLHttpRequest.
5. Custom response decoding. This is required by a server but handled by a browser natively.
6. Response is missing arrayBuffer() and blob() handlers. Headers is missing a forEach() method. These are required by the Fetch browser specification, but clearly not required for a server.

This list demonstrates why optimizing for the JavaScript runtime is important and extremely difficult to do in a single codebase. On the server, we get to use some of Node’s best features even though web browsers don’t support them.

2. Module loader support

We are occasionally asked to add support for one of several module loaders. These requests are usually accompanied by dramatic language like “poison the global scope”, meaning this line of code is considered offensive: if (!window.fetch) window.fetch = fetch;. This is precisely how a polyfill works! It provides a global API only if it’s missing in the browser.

JavaScript has four different module systems: AMD, Common.js, UMD, and standard ES6 modules. In the absence of a standard module system, the community built several of it own, which is great! For library authors, however, this is a disaster. Each of the competing module formats must be supported and tested with each new release. This is complexity that we are not interested in supporting for a polyfill. It is certainly reasonable to expect a library to support modules, but this isn’t a library.

If it were as simple as adding the standardized export default fetch; to the end of the file, we would absolutely provide that. But standard modules aren’t well supported yet, and we have no interest in supporting the legacy module loaders.

Summary

When receiving these requests, we point people to supported Node.js versions of fetch or try to provide debugging tips related to the user’s browser tools. We won’t be providing direct support for either Node.js or modules.

This polyfill provides excellent, well-tested, cross-browser fetch support that’s used in production. When we open source something like this, that’s the level of commitment we’re making. We just can’t make that commitment to tools we don’t use in production ourselves.

Hopefully this long-winded answer provides context around how we’re thinking about these issues.

❤️

[mislav]

We don’t use AMD at GitHub so we have no incentive to maintain support for AMD. It’s just 20 lines of code from your perspective, but for us it means committing to AMD support and maintaining this boilerplate code by updating it with future AMD updates. Also, if we supported AMD then we would have to support similar loaders that people ask us about, which means adding even more boilerplate code or various .json files to our project.

So you see, from your perspective it makes perfect sense for us to add these 20 lines of code to our project, and from ours it doesn’t so much. From our perspective it makes sense to continue maintaining this perfectly fine polyfill that does its job as intended.