Rust WebAssembly module in an ES module wrapper from wasm-pack fails to load in Next.js

What version of Next.js are you using?

11.1.2

What version of Node.js are you using?

14.17.6

What browser are you using?

Firefox, Chrome

What operating system are you using?

macOS

How are you deploying your application?

Not yet deployed

Describe the Bug

A simple Rust WebAssembly module packaged with its glue code into an ES module with wasm-pack (patched as in https://github.com/rustwasm/wasm-pack/pull/1061) loads and works just fine under webpack, as illustrated in https://github.com/webpack/webpack/pull/14313, but fails to import under Next.js. This is apparently because Next.js generates the .wasm generated at one path but then tries to load it from a different path.

Expected Behavior

I expected the npm run dev to successfully run the application and render a page with the greeting from inside the WebAssembly module.

I expected npm run build to successfully build a production distribution that would do the same.

To Reproduce

This is demonstrated in https://github.com/gthb/try-to-use-wasm-in-next.js/ — the README there recounts my circuitous path of trying to get this to work, but the current state serves to illustrate the problem I’m reporting here.

In that repo, first setup:

yarn install
yarn run build-wasm # or just copy the `pkg` from https://github.com/webpack/webpack/pull/14313 into `hi-wasm/pkg`
yarn run link-wasm

Then try yarn run dev — it fails like this:

yarn run v1.22.11
$ next dev
ready - started server on 0.0.0.0:3000, url: http://localhost:3000
info  - Using webpack 5. Reason: Enabled by default https://nextjs.org/docs/messages/webpack5
warn  - You have enabled experimental feature(s).
warn  - Experimental features are not covered by semver, and may cause unexpected or broken application behavior. Use them at your own risk.

event - compiled successfully
event - build page: /next/dist/pages/_error
wait  - compiling...
event - compiled successfully
event - build page: /
wait  - compiling...
event - compiled successfully
error - pages/index.js (22:51) @ Home
TypeError: (0 , hi_wasm__WEBPACK_IMPORTED_MODULE_4__.greeting) is not a function
  20 |         <p className={styles.description}>
  21 |           Greeting:
> 22 |           <code className={styles.code}>{ greeting("Bob") }</code>
     |                                                   ^
  23 |         </p>
  24 |
  25 |         <div className={styles.grid}>

Then try yarn run build — it fails like this:

yarn run v1.22.11
$ next build
info  - Using webpack 5. Reason: Enabled by default https://nextjs.org/docs/messages/webpack5
warn  - You have enabled experimental feature(s).
warn  - Experimental features are not covered by semver, and may cause unexpected or broken application behavior. Use them at your own risk.

info  - Checking validity of types
info  - Creating an optimized production build
info  - Compiled successfully

> Build error occurred
[Error: ENOENT: no such file or directory, open '/Users/gthb/git/try-to-use-wasm-in-next.js/.next/server/static/wasm/1905d306caca373cb9a6.wasm'] {
  type: 'Error',
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: '/Users/gthb/git/try-to-use-wasm-in-next.js/.next/server/static/wasm/1905d306caca373cb9a6.wasm'
}
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Note the path to the .wasm chunk. The actual generated .wasm chunks are:

$ find .next -iname '*.wasm'
.next/server/chunks/static/wasm/1905d306caca373cb9a6.wasm
.next/static/wasm/b9a4f4672eb576798896.wasm

So apparently the failure is that Next.js generates the .wasm chunk on the server side with an extra chunks/ path element (which presumably should not be there) and then looks for it at a path without that path element and fails to find it.

I’m guessing that the same problem is the cause of the npm run dev failure (i.e. the module imports as empty).

[sam3d]

The above workaround breaks when using experiments.layers = true because even though the module ID is named, a chunk hash is still appended to the output assets. The only way the above fix can work is if the client and server output the same filename for both the client and the sever.

I’ve created a more reliable workaround in the form of an embedded webpack plugin. This behaviour is pretty close to how Next.js internally handles the chunks directory and then walking back up with ../

// next.config.js

module.exports = {
  webpack(config, { isServer, dev }) {
    config.experiments = {
      asyncWebAssembly: true,
      layers: true,
    };

    if (!dev && isServer) {
      config.output.webassemblyModuleFilename = "chunks/[id].wasm";
      config.plugins.push(new WasmChunksFixPlugin());
    }

    return config;
  },
};

class WasmChunksFixPlugin {
  apply(compiler) {
    compiler.hooks.thisCompilation.tap("WasmChunksFixPlugin", (compilation) => {
      compilation.hooks.processAssets.tap(
        { name: "WasmChunksFixPlugin" },
        (assets) =>
          Object.entries(assets).forEach(([pathname, source]) => {
            if (!pathname.match(/\.wasm$/)) return;
            compilation.deleteAsset(pathname);

            const name = pathname.split("/")[1];
            const info = compilation.assetsInfo.get(pathname);
            compilation.emitAsset(name, source, info);
          })
      );
    });
  }
}

[sam3d]

@gthb Lord forgive me, but I’ve created a workaround:

// next.config.js
module.exports = {
  webpack(config, { isServer, dev }) {
    // Enable webassembly
    config.experiments = { asyncWebAssembly: true };

    // In prod mode and in the server bundle (the place where this "chunks" bug
    // appears), use the client static directory for the same .wasm bundle
    config.output.webassemblyModuleFilename =
      isServer && !dev ? "../static/wasm/[id].wasm" : "static/wasm/[id].wasm";

    // Ensure the filename for the .wasm bundle is the same on both the client
    // and the server (as in any other mode the ID's won't match)
    config.optimization.moduleIds = "named";

    return config;
  },
};

Then to ensure the .wasm file is always included in the client bundle (you don’t need to do this if you’re using it ONLY on the client, or in both the server and the client), place this somewhere in the root of your _app.tsx:

(function () {
  import("wasm/add.wasm");
  // Import which .wasm files you need here
});

This won’t download the file onto the client, just ensure that it’s in the bundle

This is obviously far from an ideal solution, because it still generates the server bundle at .next/server/chunks/../static/wasm/[id].wasm (i.e. .next/server/static/wasm/[id].wasm) that webpack is trying to import for the server, but instead gets the static one because it’s looking for .next/server/../static/wasm/[id].wasm (i.e. .next/static/wasm/[id].wasm)