Node.js script exits silently on some HTTPS requests

I’m using node-fetch to make requests to https://registry.npmjs.org/{package} behind a corporate proxy with HTTPS sniffing/TLS proxying (it is conceivable that this proxy may corrupt some requests a bit randomly). Some requests succeed but at some point, a request (not necessarily always the same) fails and the script exists silently (nothing is printed on the console output, no error is thrown, exit code is 0). It happens only behind the proxy, with HTTPS requests on https://registry.npmjs.org/{package} endpoint but not on https://registry.npmjs.org/{package}/latest endpoint (whose the response body is much lighter).

Proxy	Endpoint	HTTPS or HTTP	OK
*	/{package}/latest	*	✔️
✖️	*	*	✔️
✔️	/{package}	HTTP	✔️
✔️	/{package}	HTTPS	❌

Reproduction

Steps to reproduce the behavior:

It may be complicated to reproduce this behavior as I managed to observe it only behind my corporate proxy.

Run this script behind a buggy (😕) proxy:

const fetch = require("node-fetch");
process.env["NODE_TLS_REJECT_UNAUTHORIZED"] = 0;
(async () => {
  const packages = ["core", "animations", "common", "compiler", "core", "forms",
    "platform-browser", "platform-browser-dynamic", "platform-server", "router"];
  for (const p of packages) {
    console.log(p);
    const path = encodeURIComponent("@angular/" + p);
    const res = await fetch(`https://registry.npmjs.org/${path}`); // <-- It sometimes fails during this statement
    console.log((await res.json())["dist-tags"]["latest"]);
  }
})();

Expected behavior

node-fetch should throw an error (that I should be able to catch to retry the request).

Screenshots

PS> node .\bug.js
core
12.0.3
animations # <-- It failed silently on the second request

PS> echo $LASTEXITCODE
0

With $env:NODE_DEBUG="cluster,net,http,fs,tls,module,timers"

PS> node .\bug.js
[...]
core # <-- ✔️ this one will succeed
HTTP 17156: call onSocket 0 0
HTTP 17156: createConnection registry.npmjs.org:443::::::::::::::::::::: {
  protocol: 'https:',
  slashes: true,
  auth: null,
  host: 'registry.npmjs.org',
  port: 443,
  hostname: 'registry.npmjs.org',
  hash: null,
  search: null,
  query: null,
  pathname: '/%40angular%2Fcore',
  path: null,
  href: 'https://registry.npmjs.org/%40angular%2Fcore',
  method: 'GET',
  headers: [Object: null prototype] {
    Accept: [ '*/*' ],
    'User-Agent': [ 'node-fetch/1.0 (+https://github.com/bitinn/node-fetch)' ],
    'Accept-Encoding': [ 'gzip,deflate' ],
    Connection: [ 'close' ]
  },
  agent: undefined,
  _defaultAgent: Agent {
    _events: [Object: null prototype] {
      free: [Function (anonymous)],
      newListener: [Function: maybeEnableKeylog]
    },
    _eventsCount: 2,
    _maxListeners: undefined,
    defaultPort: 443,
    protocol: 'https:',
    options: { path: null },
    requests: {},
    sockets: { 'registry.npmjs.org:443:::::::::::::::::::::': [] },
    freeSockets: {},
    keepAliveMsecs: 1000,
    keepAlive: false,
    maxSockets: Infinity,
    maxFreeSockets: 256,
    scheduling: 'lifo',
    maxTotalSockets: Infinity,
    totalSocketCount: 0,
    maxCachedSessions: 100,
    _sessionCache: { map: {}, list: [] },
    [Symbol(kCapture)]: false
  },
  servername: 'registry.npmjs.org',
  _agentKey: 'registry.npmjs.org:443:::::::::::::::::::::'
}
TLS 17156: client _init handle? true
NET 17156: pipe false null
NET 17156: connect: find host registry.npmjs.org
NET 17156: connect: dns options { family: undefined, hints: 0 }
HTTP 17156: sockets registry.npmjs.org:443::::::::::::::::::::: 1 1
HTTP 17156: outgoing message end.
(node:17156) Warning: Setting the NODE_DEBUG environment variable to 'http' can expose sensitive data (such as passwords, tokens and authentication headers) in the resulting log.
(Use `node --trace-warnings ...` to show where the warning was created)
(node:17156) Warning: Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certificate verification.
TLS 17156: client initRead handle? true buffered? false
NET 17156: _read
NET 17156: _read wait for connection
NET 17156: afterConnect
TLS 17156: client _start handle? true connecting? false requestOCSP? false
NET 17156: _read
NET 17156: Socket._handle.readStart
TLS 17156: client emit session
TLS 17156: client onhandshakedone
TLS 17156: client _finishInit handle? true alpn false servername registry.npmjs.org
TLS 17156: client emit secureConnect. authorized: true
HTTP 17156: requestTimeout timer moved to req
HTTP 17156: AGENT incoming response!
NET 17156: _read
[... 51 _read ...]
NET 17156: _read
NET 17156: Socket._handle.readStart
NET 17156: _read
NET 17156: _read
NET 17156: Socket._handle.readStart
NET 17156: _read
NET 17156: _read
NET 17156: _read
NET 17156: Socket._handle.readStart
NET 17156: _read
NET 17156: _read
NET 17156: _read
NET 17156: Socket._handle.readStart
NET 17156: _final: not ended, call shutdown()
NET 17156: afterShutdown destroyed=false ReadableState {
  objectMode: false,
  highWaterMark: 16384,
  buffer: BufferList { head: null, tail: null, length: 0 },
  length: 0,
  pipes: [],
  flowing: true,
  ended: true,
  endEmitted: true,
  reading: false,
  sync: false,
  needReadable: false,
  emittedReadable: false,
  readableListening: false,
  resumeScheduled: false,
  errorEmitted: false,
  emitClose: false,
  autoDestroy: false,
  destroyed: false,
  errored: null,
  closed: false,
  closeEmitted: false,
  defaultEncoding: 'utf8',
  awaitDrainWriters: null,
  multiAwaitDrain: false,
  readingMore: false,
  decoder: null,
  encoding: null,
  [Symbol(kPaused)]: false
}
NET 17156: readableState ended, destroying
NET 17156: destroy
NET 17156: close
NET 17156: close handle
NET 17156: emit close
HTTP 17156: CLIENT socket onClose
HTTP 17156: removeSocket registry.npmjs.org:443::::::::::::::::::::: writable: false
HTTP 17156: HTTP socket close
core 12.0.3 # <-- ✔️ ok
animations
[...]
animations 12.0.3 # <-- ✔️ ok
common
[...]
common 12.0.3 # <-- ✔️ ok
compiler
[...]
compiler 12.0.3 # <-- ✔️ ok
core
[...]
core 12.0.3 # <-- ✔️ ok
forms # <-- ❌ this one will fail !
HTTP 17156: call onSocket 0 0
HTTP 17156: createConnection registry.npmjs.org:443::::::::::::::::::::: {
  protocol: 'https:',
  slashes: true,
  auth: null,
  host: 'registry.npmjs.org',
  port: 443,
  hostname: 'registry.npmjs.org',
  hash: null,
  search: null,
  query: null,
  pathname: '/%40angular%2Fforms',
  path: null,
  href: 'https://registry.npmjs.org/%40angular%2Fforms',
  method: 'GET',
  headers: [Object: null prototype] {
    Accept: [ '*/*' ],
    'User-Agent': [ 'node-fetch/1.0 (+https://github.com/bitinn/node-fetch)' ],
    'Accept-Encoding': [ 'gzip,deflate' ],
    Connection: [ 'close' ]
  },
  agent: undefined,
  _defaultAgent: Agent {
    _events: [Object: null prototype] {
      free: [Function (anonymous)],
      newListener: [Function: maybeEnableKeylog]
    },
    _eventsCount: 2,
    _maxListeners: undefined,
    defaultPort: 443,
    protocol: 'https:',
    options: { path: null },
    requests: {},
    sockets: { 'registry.npmjs.org:443:::::::::::::::::::::': [] },
    freeSockets: {},
    keepAliveMsecs: 1000,
    keepAlive: false,
    maxSockets: Infinity,
    maxFreeSockets: 256,
    scheduling: 'lifo',
    maxTotalSockets: Infinity,
    totalSocketCount: 0,
    maxCachedSessions: 100,
    _sessionCache: { map: [Object], list: [Array] },
    [Symbol(kCapture)]: false
  },
  servername: 'registry.npmjs.org',
  _agentKey: 'registry.npmjs.org:443:::::::::::::::::::::'
}
TLS 17156: client _init handle? true
NET 17156: pipe false null
NET 17156: connect: find host registry.npmjs.org
NET 17156: connect: dns options { family: undefined, hints: 0 }
HTTP 17156: sockets registry.npmjs.org:443::::::::::::::::::::: 1 1
HTTP 17156: outgoing message end.
TLS 17156: client initRead handle? true buffered? false
NET 17156: _read
NET 17156: _read wait for connection
NET 17156: afterConnect
TLS 17156: client _start handle? true connecting? false requestOCSP? false
NET 17156: _read
NET 17156: Socket._handle.readStart
TLS 17156: client onhandshakedone
TLS 17156: client _finishInit handle? true alpn false servername registry.npmjs.org
TLS 17156: client emit secureConnect. authorized: true
HTTP 17156: requestTimeout timer moved to req
HTTP 17156: AGENT incoming response!
NET 17156: _read
[... 56 _read ...]
NET 17156: _read
NET 17156: Socket._handle.readStart
NET 17156: _read
NET 17156: _read
NET 17156: _read
NET 17156: _read
NET 17156: _read
NET 17156: Socket._handle.readStart
NET 17156: _read
NET 17156: _read
NET 17156: _read
NET 17156: _read
NET 17156: Socket._handle.readStart
NET 17156: _read
NET 17156: _read
NET 17156: Socket._handle.readStart
NET 17156: _read
NET 17156: Socket._handle.readStart
NET 17156: _read
NET 17156: Socket._handle.readStart
NET 17156: _read
NET 17156: _read
NET 17156: _read
NET 17156: _read
NET 17156: destroy
NET 17156: close
NET 17156: close handle
NET 17156: _final: not yet connected
NET 17156: emit close
HTTP 17156: CLIENT socket onClose
HTTP 17156: removeSocket registry.npmjs.org:443::::::::::::::::::::: writable: false
HTTP 17156: HTTP socket close

# ❌ Fails silently

PS> echo $LASTEXITCODE
0

Your Environment

software	version
node-fetch	2.6.1
node	14.17.0 / 16.3.0
npm	6.14.13 / 7.15.1
Operating System	Windows 10 1909

Additional context

I had the same problem with axios.

[jimmywarting]

There are some variants to load the esm with async import wrappers in https://github.com/node-fetch/node-fetch/issues/1279#issuecomment-915060754 from cjs that i can recommend

You don’t necessary have to convert your hole project to ESM just b/c we did it.

[SgtPooki]

If you’re here and on the latest version, check out https://github.com/node-fetch/node-fetch/issues/1131 as well.