The case for Workers

Do you want to request a feature or report a bug?

Feature! 🎉

What is the current behavior?

(This is gonna be a bit longer.)

Let’s imagine working on a project using WebWorkers. Let’s say I am using this great new shiny library Comlink. It’s needed by both main.js and worker.js as Comlink is an abstraction over postMessage. Of course both main.js and worker.js have other dependencies besides Comlink.

(You might already be able to tell where I’m going with this)

Attempt 1: Entry points

Listing both worker.js and main.js as entry points doesn’t yield the expected results as all static imports will be inlined. Comlink will be (down)loaded twice. The semantics of entry points are that only one entry point will be loaded at a time. In the context of my example, though, two entry points are loaded in parallel.

Attempt 2: Dynamic loading

I tried some hackery with import(), but the problem is that WebPack (rightly) assumes that there’s is a global registry of loaded modules. worker.js, however, runs in a different realm and has a separate global scope to main.js

Workaround: Force a common chunk

My current workaround is to use the entry point approach and manually maintain a list of shared modules that need to be put in a separate chunk using CommonChunksPlugin:

module.exports = {
  entry: {
    main: './src/main.js',
    worker: './src/worker.js',
  },
  output: {
    filename: '[name].js',
  },
  plugins: [
    // Force comlink into its own bundle
    new webpack.optimize.CommonsChunkPlugin({
      name: 'comlink',
      minChunks: 2,
    }),
  ],
};

If the current behavior is a bug, please provide the steps to reproduce.

What is the expected behavior?

Under the assumption that there’s good caching headers, the optimal behavior is to put Comlink in it’s own chunk (or a chunk for shared modules) so it’s only downloaded once. The second request by the worker will hit the HTTP cache and re-use the already downloaded shared chunk.

API

I’m not sure what the API for this would look like. This API would not only be useful for workers, but also for pages that load another entry point in an iframe (although, tbh, I struggle to find a good example for this 🤷‍♂️ )

I think the most straight-forward way would to add a realmEndpoints (name pending lol) to the webpack config which lists entry points that can be loaded in parallel to one of the main entry points.

What do y’all think? API suggestions/corrections? Am I just holding it wrong?

If this is a feature request, what is motivation or use case for changing the behavior?

Please mention other relevant information such as the browser version, Node.js version, webpack version and Operating System.

[surma]

Currently a target: “web” compilation and a target: “webworker” compilation can’t share chunks, because they are using different ways to lazy-load chunks. web uses JSONP, webworker uses importScripts.

I forgot about that bit! In my workaround (see below) I currently also hand-load the chunks using importScript. I think an agnostic format that can be loaded by both workers and web would be great to see.

const base = `${location.protocol}//${location.host}${location.pathname.split('/').slice(0, -1).join('/')}`;
new Worker(`data:application/javascript,window=self;importScripts('${base}/comlink.js');importScripts('${base}/worker.js');`));

[terpimost]

@sokra and @anilanar thank very much for clarifications. I understand that new way of creating a worker is const w = new Worker(new URL('./worker.js', import.meta.url)); which will handle things for us, but could you please clarify how can I do the same without using import.meta.url? From my understand steps should be like this:

• allow webpack to split chunks chunks: 'all', minSize: 0,
• create and entry point for the worker to actually produce it as a file
• exclude such worker in HtmlWebpackPlugin so it will not be placed in index.html
• create a worker as with usual file I’m confused about the last step, const w = new Worker('worker.js') and const w = new Worker('./worker.js') doesn’t work.

I guess I should use a new way of importing assets in Webpack 5 or somehow construct the URL without using import.meta.url. Or I just should use old WorkerLoader for this purpose?

Could you please help me with that?