(lambda-nodejs): Unable to use some packages since move to esbuild

This is a continuation of AWS support case 8536830561, where I was advised to open the issue here as it appears to be a bug introduced in a recent CDK change.

Since the move to esbuild, stacks that previously deployed without issue will no longer build. This seems to affect any Lambda function that uses a module that requires a compilation step.

Reproduction Steps

npm install https://github.com/adam-nielsen/netcdf4

Add to a Lambda:

const netcdf4 = require('netcdf4');
let cdf = new netcdf4.File('dummy.cdf', 'r');

Try to build it:

cdk synth
> asset-input/test.js:1:24: error: No loader is configured for ".node" files: asset-input/node_modules/netcdf4/build/Release/netcdf4.node
    1 │ const netcdf4 = require('netcdf4');

Note the module being installed is my own fork of an upstream one modified to compile in a more recent version of Node. I am happy to further modify it to get it to work with esbuild, if that’s an option.

There is a minimal working example here: http://uq-ads-aws-support-public.s3-website-ap-southeast-2.amazonaws.com/cdk-bundle-test.zip

It can be run like so:

npm install
cd lambda
npm install
cd ..
cdk synth

What did you expect to happen?

I expected it to build and deploy as it did with CDK 1.73.0.

What actually happened?

It failed with an error:

> asset-input/test.js:1:24: error: No loader is configured for ".node" files: asset-input/node_modules/netcdf4/build/Release/netcdf4.node
    1 │ const netcdf4 = require('netcdf4');

Environment

• CDK CLI Version : 1.109.0
• Framework Version: 1.110.1
• Node.js Version: v16.2.0
• OS : Arch Linux
• Language (Version): Javascript

Other

There appears to be some discussion upstream about this issue however I am not sure whether that proposed fix can be put into CDK.

---

This is 🐛 Bug Report

[privogpynes]

I had this issue a while back deploying lambda@edge with the sharp module as it also has native dependencies. You’re spot on about the compiled native outputs, they’re binaries which is why you can’t actually bundle them into your js the way esbuild wants.

I took a stab at solving your issue using the sample you provided. When I tried to download that talib package you had though, even trying to just npm install it locally gave me a bunch of install/compile errors, same with your netcdf4 package. So I replaced them with sharp and took a look at your lib/cdk-bundle-test-stack.js and noticed you didn’t have the nodeModules property inside the bundling options for your NodejsFunction. I popped it in there and also turned on the forceDockerBundling option so that the native libs get built using the the proper lambda os environment instead of my computer’s os and everything works fine. I don’t think you need the forceDockerBundling since esbuild wasn’t in your dev deps, but tossed in regardless.

Here’s the version I got working. Not sure if this helps you out or not cdk-bundle-test.zip

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.