Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Express with Webpack not sending Transactions

See original GitHub issue

Describe the bug

I have an express service that bundled with Webpack and runs on Docker (it’s mean that in the Dockerfile you just need CMD ["node", "--enable-source-maps", "main.js"]

To Reproduce

Steps to reproduce the behavior:

  1. Create a simple Express service.
  2. Add import apm from 'elastic-apm-node/start' at the top on the file.
  3. Create a webpack.config.js file:
const { CleanWebpackPlugin } = require('clean-webpack-plugin')

module.exports = {

    target: 'node',
    mode: 'production',
    devtool: 'source-map',
    module: {
        rules: [
            {
                test: /\.js$/,
                include: __dirname + '/src',
                use: 'babel-loader'
            }
        ]
    },
    plugins: [
        new CleanWebpackPlugin()
    ]
}
  1. Use these variables:
ELASTIC_APM_SERVER_URL=http://apm-server:8200
ELASTIC_APM_VERIFY_SERVER_CERT=false
ELASTIC_APM_LOG_LEVEL=trace
ELASTIC_APM_SERVICE_NAME=service-name

Expected behavior

When I’m running in debug mode, without webpack, by babel-node --inspect=33180 src/index.js, I’m getting the transactions as expected.

Here is the configuration output from Debug mode:

agent configured correctly {
  pid: 21432,
  ppid: 17216,
  arch: 'x64',
  platform: 'win32',
  node: 'v12.18.2',
  agent: '3.10.0',
  ancestors: [
    'absolutePath\\service-name\\node_modules\\elastic-apm-node\\index.js',
    'absolutePath\\service-name\\node_modules\\elastic-apm-node\\start.js',
    'absolutePath\\service-name\\src\\index.js',
    [length]: 3
  ],
  startTrace: [
    'at Agent.start (absolutePath\\service-name\\node_modules\\elastic-apm-node\\lib\\agent.js:154:11)',
    'at Object.<anonymous> (absolutePath\\service-name\\node_modules\\elastic-apm-node\\start.js:3:32)',
    'at Module._compile (internal/modules/cjs/loader.js:1138:30)',
    'at Module._compile (absolutePath\\service-name\\node_modules\\pirates\\lib\\index.js:99:24)',
    'at Module._extensions..js (internal/modules/cjs/loader.js:1158:10)',
    'at Object.newLoader [as .js] (absolutePath\\service-name\\node_modules\\pirates\\lib\\index.js:104:7)',
    'at Module.load (internal/modules/cjs/loader.js:986:32)',
    'at Function.Module._load (internal/modules/cjs/loader.js:879:14)',
    'at Module.require (internal/modules/cjs/loader.js:1026:19)',
    'at require (internal/modules/cjs/helpers.js:72:18)',
    [length]: 10
  ],
  main: undefined,
  dependencies: undefined,
  conf: {
    ignoreUrlStr: [ [length]: 0 ],
    ignoreUrlRegExp: [ [length]: 0 ],
    ignoreUserAgentStr: [ [length]: 0 ],
    ignoreUserAgentRegExp: [ [length]: 0 ],
    transactionIgnoreUrlRegExp: [ [length]: 0 ],
    sanitizeFieldNamesRegExp: [
      /^password$/i { [lastIndex]: 0 },
      /^passwd$/i { [lastIndex]: 0 },
      /^pwd$/i { [lastIndex]: 0 },
      /^secret$/i { [lastIndex]: 0 },
      /^.*key$/i { [lastIndex]: 0 },
      /^.*token.*$/i { [lastIndex]: 0 },
      /^.*session.*$/i { [lastIndex]: 0 },
      /^.*credit.*$/i { [lastIndex]: 0 },
      /^.*card.*$/i { [lastIndex]: 0 },
      /^authorization$/i { [lastIndex]: 0 },
      /^set\x2dcookie$/i { [lastIndex]: 0 },
      /^pw$/i { [lastIndex]: 0 },
      /^pass$/i { [lastIndex]: 0 },
      /^connect\.sid$/i { [lastIndex]: 0 },
      [length]: 14
    ],
    abortedErrorThreshold: 25,
    active: true,
    apiRequestSize: 786432,
    apiRequestTime: 10,
    asyncHooks: true,
    breakdownMetrics: true,
    captureBody: 'off',
    captureErrorLogStackTraces: 'messages',
    captureExceptions: true,
    captureHeaders: true,
    captureSpanStackTraces: true,
    centralConfig: true,
    disableInstrumentations: [ [length]: 0 ],
    environment: 'development',
    errorMessageMaxLength: 2048,
    errorOnAbortedRequests: false,
    filterHttpHeaders: true,
    instrument: true,
    instrumentIncomingHTTPRequests: true,
    logLevel: 'trace',
    logUncaughtExceptions: false,
    metricsInterval: 30,
    metricsLimit: 1000,
    sanitizeFieldNames: [
      'password',   'passwd',
      'pwd',        'secret',
      '*key',       '*token*',
      '*session*',  '*credit*',
      '*card*',     'authorization',
      'set-cookie', 'pw',
      'pass',       'connect.sid',
      [length]: 14
    ],
    serverTimeout: 30,
    sourceLinesErrorAppFrames: 5,
    sourceLinesErrorLibraryFrames: 5,
    sourceLinesSpanAppFrames: 0,
    sourceLinesSpanLibraryFrames: 0,
    stackTraceLimit: 50,
    transactionIgnoreUrls: [ [length]: 0 ],
    transactionMaxSpans: 500,
    transactionSampleRate: 1,
    useElasticTraceparentHeader: true,
    usePathAsTransactionName: false,
    verifyServerCert: false,
    serverUrl: 'http://apm-server:8200',
    serviceName: 'service-name',
    serviceVersion: '1.0.0',
    serverHost: 'apm-server:8200',
    serverPort: 8200
  }
}
 node_modules/console-log-level/index.js
adding hook to Node.js module loader
 node_modules/console-log-level/index.js
shimming http@12.18.2 module
 node_modules/console-log-level/index.js
shimming http.Server.prototype.emit function
 node_modules/console-log-level/index.js
shimming http.ServerResponse.prototype.writeHead function
 node_modules/console-log-level/index.js
shimming http.request function
 node_modules/console-log-level/index.js
shimming http.get function
 node_modules/console-log-level/index.js
shimming https@12.18.2 module
 node_modules/console-log-level/index.js
shimming https.Server.prototype.emit function
 node_modules/console-log-level/index.js
shimming https.request function
 node_modules/console-log-level/index.js
shimming https.get function
 node_modules/console-log-level/index.js
shimming elasticsearch@15.5.0 module
 node_modules/console-log-level/index.js
shimming elasticsearch.Transport.prototype.request
 node_modules/console-log-level/index.js
shimming finalhandler@1.1.2 module
 node_modules/console-log-level/index.js
shimming express@4.17.1 module
 node_modules/console-log-level/index.js
shimming express.Router.use function
 node_modules/console-log-level/index.js
shimming express.static function
 node_modules/console-log-level/index.js
copying property mime from express.static
 node_modules/console-log-level/index.js
shimming mongodb@3.6.0 module
 node_modules/console-log-level/index.js
shimming express.Router.Layer.handle function: query
 node_modules/console-log-level/index.js
shimming express.Router.Layer.handle function: expressInit
 node_modules/console-log-level/index.js
shimming express.Router.Layer.handle function: compression
 node_modules/console-log-level/index.js
shimming express.Router.Layer.handle function: corsMiddleware
 node_modules/console-log-level/index.js
shimming express.Router.Layer.handle function: bound dispatch
shimming express.Router.Layer.handle function: bound dispatch
shimming express.Router.Layer.handle function: bound dispatch
shimming express.Router.Layer.handle function: bound dispatch
shimming express.Router.Layer.handle function: bound dispatch
shimming express.Router.Layer.handle function: bound dispatch
shimming express.Router.Layer.handle function: bound dispatch
 node_modules/console-log-level/index.js

Actual behavior

But in production, I can see the service, I even capturing errors manually, but I can’t see the transactions.

Here is the config output from the production (webpack) run:

agent configured correctly {

,
  ppid: 0,
  arch: 'x64',
  platform: 'linux',
  node: 'v12.19.1',
  agent: '3.10.0',
  ancestors: [ [length]: 0 ],
  startTrace: [
    'at v.start (/usr/src/app/main.js:200:3475)',
    'at Object.<anonymous> (/usr/src/app/main.js:200:67)',
    'at n (/usr/src/app/main.js:1:110)',
    'at Module.<anonymous> (/usr/src/app/main.js:384:219812)',
    'at n (/usr/src/app/main.js:1:110)',
    'at /usr/src/app/main.js:1:902',
    'at Object.<anonymous> (/usr/src/app/main.js:1:913)',
    'at Module._compile (internal/modules/cjs/loader.js:1015:30)',
    'at Object.Module._extensions..js (internal/modules/cjs/loader.js:1035:10)',
    'at Module.load (internal/modules/cjs/loader.js:879:32)',
    [length]: 10
  ],
  main: undefined,
  dependencies: undefined,

{
    ignoreUrlStr: [ [length]: 0 ],
    ignoreUrlRegExp: [ [length]: 0 ],
    ignoreUserAgentStr: [ [length]: 0 ],
    ignoreUserAgentRegExp: [ [length]: 0 ],
    transactionIgnoreUrlRegExp: [ [length]: 0 ],
    sanitizeFieldNamesRegExp: [
      /^password$/i { [lastIndex]: 0 },
      /^passwd$/i { [lastIndex]: 0 },
      /^pwd$/i { [lastIndex]: 0 },
      /^secret$/i { [lastIndex]: 0 },
      /^.*key$/i { [lastIndex]: 0 },
      /^.*token.*$/i { [lastIndex]: 0 },
      /^.*session.*$/i { [lastIndex]: 0 },
      /^.*credit.*$/i { [lastIndex]: 0 },
      /^.*card.*$/i { [lastIndex]: 0 },
      /^authorization$/i { [lastIndex]: 0 },
      /^set\x2dcookie$/i { [lastIndex]: 0 },
      /^pw$/i { [lastIndex]: 0 },
      /^pass$/i { [lastIndex]: 0 },
      /^connect\.sid$/i { [lastIndex]: 0 },
      [length]: 14
    ],
    abortedErrorThreshold: 25,
    active: true,
    apiRequestSize: 786432,
    apiRequestTime: 10,
    asyncHooks: true,
    breakdownMetrics: true,
    captureBody: 'off',
    captureErrorLogStackTraces: 'messages',
    captureExceptions: true,
    captureHeaders: true,
    captureSpanStackTraces: true,
    centralConfig: true,
    disableInstrumentations: [ [length]: 0 ],
    environment: 'production',
    errorMessageMaxLength: 2048,
    errorOnAbortedRequests: false,
    filterHttpHeaders: true,
    instrument: true,
    instrumentIncomingHTTPRequests: true,
    logLevel: 'trace',
    logUncaughtExceptions: false,
    metricsInterval: 30,
    metricsLimit: 1000,
    sanitizeFieldNames: [
      'password',   'passwd',
      'pwd',        'secret',
      '*key',       '*token*',
      '*session*',  '*credit*',
      '*card*',     'authorization',
      'set-cookie', 'pw',
      'pass',       'connect.sid',
      [length]: 14
    ],
    serverTimeout: 30,
    sourceLinesErrorAppFrames: 5,
    sourceLinesErrorLibraryFrames: 5,
    sourceLinesSpanAppFrames: 0,
    sourceLinesSpanLibraryFrames: 0,
    stackTraceLimit: 50,
    transactionIgnoreUrls: [ [length]: 0 ],
    transactionMaxSpans: 500,
    transactionSampleRate: 1,
    useElasticTraceparentHeader: true,
    usePathAsTransactionName: false,
    verifyServerCert: false,
    serverUrl: 'http://apm-server:8200',
    serviceName: 'service-name-dev-env',
    serverHost: 'apm-server:8200',
    serverPort: 8200
  }
}
adding hook to Node.js module loader
shimming https@12.19.1 module
shimming https.Server.prototype.emit function
shimming https.request function
shimming https.get function

Please note there are no ancestors, and there are fewer shimming. Also, note I named the “production” (webpack) service with a different name ('service-name-dev-env) to identify it.

Environment (please complete the following information)

You can see the config output. It is better.

APM Server version:

{
    "build_date": "2019-03-06T14:11:42Z",
    "build_sha": "daad5449d758e7bf101f6482778a2b15f646a67f",
    "version": "6.6.2"
}

How are you starting the agent? (please tick one of the boxes)

  • Calling agent.start() directly (e.g. require('elastic-apm-node').start(...))
  • Requiring elastic-apm-node/start from within the source code
  • Starting node with -r elastic-apm-node/start

Additional context

  • package.json dependencies:

    Click to expand 
    "dependencies": {
    "axios": "^0.18.1",
    "compression": "^1.7.4",
    "cors": "^2.8.5",
    "elastic-apm-node": "^3.10.0",
    "elasticsearch": "^15.4.1",
    "express": "^4.17.1",
    "http-status-codes": "^1.4.0",
    "mongodb": "^3.5.2"
},
"devDependencies": {
    "@babel/core": "^7.10.4",
    "@babel/preset-env": "^7.10.4",
    "babel-loader": "^8.1.0",
    "clean-webpack-plugin": "^3.0.0",
    "webpack": "^4.43.0",
    "webpack-cli": "^3.3.12"
},
"optionalDependencies": {
    "@babel/cli": "^7.10.4",
    "@babel/node": "^7.10.4",
    "@team17/eslint-config": "^2.1.0",
    "babel-eslint": "^10.1.0",
    "babel-jest": "^26.3.0",
    "eslint": "^6.8.0",
    "eslint-friendly-formatter": "^4.0.1",
    "eslint-plugin-babel": "^5.3.1",
    "eslint-plugin-jest": "^23.20.0",
    "husky": "^4.2.5",
    "jest": "^26.4.2"
}

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:14 (7 by maintainers)

github_iconTop GitHub Comments

2reactions
baruchirocommented, Feb 2, 2021

Off-topic- why do we use Webpack for the backend?

I thought to write an article about that.

Anyway, we have two use-cases. For our discussion, we are talking about a backend service with Express. We are running that service in a Docker, and we have a lot of services (micro-services).

To reduce the Docker Image size, we are using Webpack to tree-shaking the dependencies and so on, so that we getting a minimal Docker Image:

# nodejs binaries
FROM node:12-alpine as binaries

# builder image
FROM node:12 as builder

WORKDIR /usr/src/app
COPY . .

RUN npm install
RUN npm run build

# release image
FROM alpine

COPY --from=binaries /usr/local/bin/node /usr/local/bin/
COPY --from=binaries /usr/lib/ /usr/lib/

WORKDIR /usr/src/app
COPY --from=builder /usr/src/app/dist .

CMD ["node", "--enable-source-maps", "main.js"]
1reaction
gtorrecommented, Dec 10, 2021

@trentm huge thank you for your help on this. We had a similar issue to @baruchiro. Loading the agent first in the Dockerfile did the trick! I’m hopeful https://github.com/elastic/apm-agent-nodejs/issues/1967 is merged soon 🤞

Read more comments on GitHub >

github_iconTop Results From Across the Web

How can I use webpack with express? - Stack Overflow
What I ended up doing was I used 2 different configurations, 1 for packing the server stuff together using webpack, and 1 for...
Read more >
Creating a Node Express-Webpack App with Dev and Prod ...
We need the CLI tools to call it from the command line, and we need webpack-node-externals to ignore node_modules when creating the build...
Read more >
Elastic node apm not adding hooks for express
Hi, I am using express 4.16.2 version. To use node APM , we added following line at the start of code in index.js...
Read more >
Node Agent fails with WebPack - New Relic Explorers Hub
Webpack should not be used with the node agent as part of an isomorphic application. It will result in the agent not working...
Read more >
Nodejs Security - OWASP Cheat Sheet Series
If there is no limit on the size of requests, attackers can send requests with ... You can accomplish this very easily with...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

Reconsider structure of `context.db.statement`

Next page

Add filtering support for metadata payload