When I add an external library (as a directory), there is an error installing dependencies of the package.

This is a (Bug Report)

Description

For bug reports:

• What went wrong? I’m using serverless-webpack with a project that uses a lambda layer made with typescript for common modules and code. We are deploying the layer in remote and in local in “/opt” directory so i want to exclude this directories on the Webpack bundle.

For doing so, I’ve added our directory as a regular expression to the externals declaration of Webpack config. This works fine, as the dependency is treated as external, but when is trying to install dependencies it generates a ‘empty’ package.json package name & package version and the npm install to for creating dependencies fails.

On line 128 of packExternalModules.js in serverless-webpack package is where it tries to push the module without name or version.

      if (!packageJson.devDependencies || !packageJson.devDependencies[module.external]) {
        // Add transient dependencies if they appear not in the service's dev dependencies
        const originInfo = _.get(dependencyGraph, 'dependencies', {})[module.origin] || {};
        moduleVersion = _.get(_.get(originInfo, 'dependencies', {})[module.external], 'version');
        if (!moduleVersion) {
          this.serverless.cli.log(`WARNING: Could not determine version of module ${module.external}`);
        }
        prodModules.push(moduleVersion ? `${module.external}@${moduleVersion}` : module.external);

I’ve used a workaround with forceExclude in serverless.yml

custom:
  webpack:
    webpackConfig: ./webpack.config.js
    includeModules:
      forceExclude:
        - aws-sdk
        ## Fix to not try to deploy dependencies using webpack.config.js ^\/opt\/layer-utils\// regex
        - ''

• What did you expect should have happened? As it is an external directories with my own packages, I expect to be identified as that and don’t try to install them. Also detect the name and version correctly.
• What was the config you used? This is my Webpack.config.js file

const path = require('path')
const webpack = require('webpack')
const slsw = require('serverless-webpack')
const nodeExternals = require('webpack-node-externals')
const ForkTsCheckerWebpackPlugin = require('fork-ts-checker-webpack-plugin')

module.exports = {
    context: __dirname,
    mode: slsw.lib.webpack.isLocal ? 'development' : 'production',
    entry: slsw.lib.entries,
    devtool: slsw.lib.webpack.isLocal
        ? 'cheap-module-eval-source-map'
        : 'source-map',
    resolve: {
        extensions: ['.mjs', '.json', '.ts','.js', '.jsx'],
        symlinks: false,
        cacheWithContext: false
    },
    output: {
        libraryTarget: 'commonjs',
        path: path.join(__dirname, 'dist'),
        filename: '[name].js'
    },
    target: 'node',
    externals: [nodeExternals(),
                function(context, request, callback) {
                    if (/^\/opt\/layer-utils\//.test(request)){
                    return callback(null, 'commonjs ' + request);
                    }
                    callback();
                }      
            ],
    module: {
        rules: [
            // all files with a `.ts` or `.tsx` extension will be handled by `ts-loader`
            {
                test: /\.(tsx?)$/,
                loader: 'ts-loader',
                exclude: [
                    [
                        path.resolve(__dirname, 'node_modules'),
                        path.resolve(__dirname, '.serverless'),
                        path.resolve(__dirname, '.webpack'),
                        path.resolve('/opt')
                    ]
                ],
                options: {
                    transpileOnly: true,
                    experimentalWatchApi: true
                }
            }
        ]
    },
    plugins: [

    ]
}

• What stacktrace or error message from your provider did you see?

Serverless: Fetch dependency graph from /app/package.json
Serverless: WARNING: Could not determine version of module 
Serverless: WARNING: Could not determine version of module 
Serverless: Excluding external modules: aws-sdk@^2.664.0
Serverless: Package lock found - Using locked versions
Serverless: Packing external modules: 
 
  Error --------------------------------------------------
 
  Error: npm install failed with code 1
      at ChildProcess.<anonymous> (/app/node_modules/serverless-webpack/lib/utils.js:91:16)
      at ChildProcess.emit (events.js:310:20)
      at ChildProcess.EventEmitter.emit (domain.js:482:12)
      at maybeClose (internal/child_process.js:1021:16)
      at Process.ChildProcess._handle.onexit (internal/child_process.js:286:5)
 
     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.
 
  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com

And it has created the file ./dist/dependencies/package.json with this content:

{
  "name": "MyName",
  "version": "1.0.0",
  "description": "MyDescription",
  "private": true,
  "scripts": {},
  "dependencies": {
    "": ""
  }

For feature proposals:

• What is the use case that should be solved. The more detail you describe this in the easier it is to understand for us.
• If there is additional config how would it look

It should be an easier way to exclude directories without try to install it as a dependency. Or add an optional boolean parameter to externals to forceExclude on dependenciesPackage

Similar or dependent issue(s):

Additional Data

• Serverless-Webpack Version you’re using: “^5.3.4”
• Webpack version you’re using: “^4.43.0”,
• Serverless Framework Version you’re using: Framework Core: 1.77.0 Plugin: 3.6.18 SDK: 2.3.1 Components: 2.33.0
• Operating System: Dockerized linux alpine
• Stack Trace (if available):

[ssanri]

same issue.

Error: npm.cmd ls -prod -json -depth=1 failed with code 1
      at ChildProcess.child.on.exitCode (C:\dev\serverless\services-legacy\assesments-service\node_modules\serverless-webpack\lib\utils.js:91:16)
      at ChildProcess.emit (events.js:189:13)
      at ChildProcess.EventEmitter.emit (domain.js:441:20)
      at maybeClose (internal/child_process.js:970:16)
      at Process.ChildProcess._handle.onexit (internal/child_process.js:259:5)

also in my case the issue is caused by a private repository in package.json with git+ssh://[e-mail]... target.

[Hideman85]

Having the same troubles right now. I’m having a module targeting a git repo and all the deps of that module are found without a version resulting in the console warn and a package.json with lot of "module": "" dependencies 🙁

Anyone found a workaround I would love to finish my feature 🙂

Edit: Could we have a fallback like require('aws-sdk/package.json').version? Edit2: There is my custom fallbacks:

if (!moduleVersion) {
  try {
    moduleVersion = require(`${module.external}/package.json`).version;
  } catch {}
}
if (!moduleVersion) {
  try {
    //  Need to remove potential nested subfolders and file from the path
    const modulePath = require.resolve(module.external).replace(new RegExp(`${module.external}/.*$`), module.external);
    moduleVersion = require(`${modulePath}/package.json`).version;
  } catch {}
}