Compatibility with bundling tools like webpack and rollup

A way to find binaries by contents of package.json files seems to be needed for compatibility with JS bundlers.

As this code in gRPC’s repo, we need to pass paths to package.json to the find(package_json_path, opts) function in lib/pre-binding.js currently. However, when the paths to package.json are hard-coded in source files like the one of gRPC, bundling tools, such as webpack and rollup, cannot detect the dynamic loading of external files and end up creating bundled files which don’t work.

So, we need some function like findByPackageInfo(package_json_content, opts). Then, we can change the code of gRPC like:

var binary = require('node-pre-gyp/lib/pre-binding');
var path = require('path');
var binding_path = binary.findByPackageInfo(require('../../../package.json'));
var binding = require(binding_path);

Any thoughts?

References

• The similar issue of gRPC
• webpack | Module Resolution

[rajivshah3]

If anyone comes here looking for a solution for Webpack, this fixed the problem for us:

I solved it by adding

externals: { 'sqlite3':'commonjs sqlite3', }

to the webpack config file

Simply replace sqlite3 with the the dependency that uses node-pre-gyp (in our case, argon2)

[dmarcs]

This pull request https://github.com/mapbox/node-pre-gyp/pull/622 solves the main problem that prevents @mapbox/node-pre-gyp from working with webpack.

webpack needs to know where package.json and the .node file are located during the build process. Right now both locations are determined at run-time instead of during the build process.

Using externals as described above is a valid workaround, but it requires that the package is available at run-time. This means that after you build with webpack, you need to run npm install package. It would be better if we could avoid npm install package since it adds an extra step.

@springmeyer a couple reasons webpack is useful include

1. Say you have a node app written in Typescript. You normally have to convert the Typescript (server.ts) to Javascript (server.js), and then run node server.js. If you make a change to server.ts you now need to recompile the Typescript to Javascript, stop server.js, and re-run server.js. With webpack you can make a change to server.ts, and you don’t need to recompile, stop the server, and re-run the server. It does all of those steps for you. You could use nodemon and typescript watch instead of webpack for this though.

2. If you have a node project with many relative imports, for example import '../../../../src/file.js' the code can become unreadable. With webpack we can get rid of every …/ and write code that looks cleaner, import 'src/file.js' in this example.

You could argue that webpack isn’t necessary (it’s much more useful for front-end development than back-end development), but you might be stuck using a node framework that depends on webpack.