Transactions with result "success" mixed with "HTTP 2xx"

If certain conditions are present, it’s possible for the user to see transactions with type request also have the result success. This result value is the default result value of transactions and should normally be overridden when it’s a transaction that’s automatically started by the agent based on an incomming HTTP request (in which case the type is set to request).

Here’s a sample program that will trigger the problem:

const agent = require('elastic-apm-node').start({
  serviceName: 'test'
})
const assert = require('assert')

// hijack the request to the APM Server to validate the payload
agent._httpClient.request = function (endpoint, headers, data, cb) {
  assert.equal(data.transactions.length, 1)
  assert.equal(data.transactions[0].result, 'HTTP 2xx')
  server.close()
}

var server = http.createServer(function (req, res) {
  // simulate that the transaction is lost before res.writeHead is called
  agent._instrumentation.currentTransaction = null
  res.end()
})

server.listen(function () {
  // send a test request to the HTTP Server to trigger the problem
  http.get('http://localhost:' + server.address().port, function (res) {
    res.resume()
    res.on('end', function () {
      agent.flush()
    })
  })
})

The part that triggers the problem is where I’m manually setting currentTransaction = null. While users should not do this IRL, it might happen if the app contains a user-land callback queue that we haven’t patched.

As such there’s nothing we can do about loosing the transaction in that case, but it’s confusing to the user to suddenly see a transactions with the result success mixed together with results like HTTP 2xx etc.

The reason why loosing the transaction triggers this problem is because of the way we set the result of the transaction:

https://github.com/elastic/apm-agent-nodejs/blob/b2ee65333947fc93cdb02b7fcbbaa8af3f2482a6/lib/instrumentation/modules/http.js#L26-L31

As you can see in this code snippet, we depend on being able to have access to the currentTransaction when res.writeHead is called.

On the other hand, we’ll automatically end the transaction when the response ends:

https://github.com/elastic/apm-agent-nodejs/blob/b2ee65333947fc93cdb02b7fcbbaa8af3f2482a6/lib/instrumentation/http-shared.js#L18-L24

So it would probably be possible for us to set the result here, either if we wasn’t able to do it when res.writeHead was called, or even dropping the logic in res.writeHead all together.

Gotcha

While this would help set the correct result on the transactions, we would not have discovered that we were loosing the context in this case if it wasn’t because we suddenly saw these weird success results. So it will hide the fact that something is wrong. It would therefore be preferable if, at the same time, we considered how we can highlight this loss of context in a way that doesn’t seem like a bug.

[watson]

@JherezTaylor FYI: Support for express-queue have been added in v1.7.1

[watson]

This have now been fixed in v1.14.0