HTTPS Requests are Failing with Node 9

It seems like there’s a regression with HTTPS urls in Node 9 that’s not there in node8. Here’s an example test run

Steps to Repeat

$ docker run  -it -v$(pwd):/app node:9-alpine /bin/sh
# inside container
$ NODE_DEBUG=net,http,https yarn test -- --grep "matching an https"

Actual Results

  Replay
    matching an https url
      1) "before all" hook


  0 passing (5s)
  1 failing

  1) Replay
       matching an https url
         "before all" hook:
     Error: Timeout of 5000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.

Expected Results

the “matching https url” should pass, which would indicate working support for HTTPS requests.

Suspicions

• debug output (see below) shows 2 separate requests
• debug logs for https proxy server are not present – maybe node is blocking
• could node9 be preventing the dns-patcher from working?

Full Debug Output

/app # NODE_DEBUG=http,https,net,tls DEBUG=replay  yarn test -- --grep 'matching an https'
NET 710: _read
NET 710: Socket._read readStart
yarn run v1.3.2
warning From Yarn 1.0 onwards, scripts don't require "--" for options to be forwarded. In a future version, any explicit "--" will be forwarded as-is to the scripts.
$ mocha --grep 'matching an https'


  Replay
    matching an https url
HTTP 737: call onSocket 0 0
HTTP 737: createConnection example.com:3443:::::::false:: { servername: 'example.com',
  _defaultAgent:
   Agent {
     domain: null,
     _events: { free: [Function] },
     _eventsCount: 1,
     _maxListeners: undefined,
     defaultPort: 443,
     protocol: 'https:',
     options: { path: null },
     requests: {},
     sockets: { 'example.com:3443:::::::false::': [] },
     freeSockets: {},
     keepAliveMsecs: 1000,
     keepAlive: false,
     maxSockets: Infinity,
     maxFreeSockets: 256,
     maxCachedSessions: 100,
     _sessionCache: { map: {}, list: [] } },
  timeout: 500,
  hostname: 'example.com',
  port: 3443,
  path: null,
  rejectUnauthorized: false,
  host: 'example.com',
  _agentKey: 'example.com:3443:::::::false::' }
HTTPS 737: createConnection { servername: 'example.com',
  _defaultAgent:
   Agent {
     domain: null,
     _events: { free: [Function] },
     _eventsCount: 1,
     _maxListeners: undefined,
     defaultPort: 443,
     protocol: 'https:',
     options: { path: null },
     requests: {},
     sockets: { 'example.com:3443:::::::false::': [] },
     freeSockets: {},
     keepAliveMsecs: 1000,
     keepAlive: false,
     maxSockets: Infinity,
     maxFreeSockets: 256,
     maxCachedSessions: 100,
     _sessionCache: { map: {}, list: [] } },
  timeout: 500,
  hostname: 'example.com',
  port: 3443,
  path: null,
  rejectUnauthorized: false,
  host: 'example.com',
  _agentKey: 'example.com:3443:::::::false::',
  encoding: null }
NET 737: pipe false null
NET 737: connect: find host example.com
NET 737: connect: dns options { family: undefined, hints: 32 }
HTTP 737: sockets example.com:3443:::::::false:: 1
HTTP 737: outgoing message end.
NET 737: _read
NET 737: _read wait for connection
      1) "before all" hook

Build Info

git log -1|cat
68cc2bc - (HEAD -> master, origin/master, origin/HEAD) Merge pull request #140 from albertogasparin/fix/ignore-dot-files (28 hours ago) <Assaf Arkin>%
 node -v
v9.2.1
/ # uname -a
Linux 10ee846c2cf3 4.9.49-moby #1 SMP Wed Sep 27 23:17:17 UTC 2017 x86_64 Linux

[g-sam]

We are using replay to mock calls to a https endpoint.

On Node 9: DEBUG=replay gives no output and the real endpoint is hit. On Node 8: it works as expected.

Let me know if you need something more comprehensive but @tonymet has already given steps to reproduce above.

[ostrgard]

The issue is reproduceable following the steps in this repo: https://github.com/ostrgard/node-replay-bug. Node’s http and https seems to work fine, but other fetching libraries have issues with https requests.

Adding HTTPS.request = HTTP.request; to the end of src/patch_http_request.js fixes this, but does fail some tests.

Hope this is of some help, @assaf.