Add ability to rename asset module for code generation (Loader API, image conversion issues)

Feature request

What is the expected behavior?

Edit: The goal is to implement new API to be able to rename the output file e.g. in an image optimization loader, which can convert the source image to different format and return that as its result (need to change [ext]). Current solutions for this situation are based on the deprecated this._module and overwriting this.resourcePath , this._module.matchResource. See the idea of the new “resourceURL” below, which could be used for code generation, if it was provided by module (i.e. loader sets it).

Be able to define custom placeholders when developing loaders. Consider these custom placeholders [width] and [newext] in an image loader plugin which can convert and resize:

{
  test: /\.(jpe?g|png|webp|avif|tiff)$/i,
  type: 'asset',
  generator: {
    filename: '[name]-[width]-[hash].[newext]'
  },
  use: [
    {
      loader: path.resolve(__dirname, 'MyImageLoader.js'),
      options: {}
    }
  ]
}

What is motivation or use case for adding/changing the behavior?

Think of a custom image loader which can resize and convert images e.g. from CSS urls like some.jpg?format=webp&width=800. Now I can only change the output file extension in my Webpack config like this:

  generator: {
    filename: (pathData) => {
      // get query params for the current file (e.g. ?format=webp)
      const info = pathData.module.resourceResolveData;
      const params = new URLSearchParams(info.query);

      // determine the new output file extension
      const format = params.get('format') || path.extname(info.path).split('.').pop();

      // default filename
      let fileName = '[name].[contenthash:8][ext]';

      // change the extension to the converted format
      format && (fileName = fileName.replace("[ext]", `.${format}`));

      return fileName;
    }
  },

Of course this should be done by the loader itself, but it can’t because this._module is deprecated and this would be an ugly hack.

Maybe it’s my lack of knowledge but currently I can’t find an elegant way to let the converted image file emitted with the new extension and at the same time properly rewrite the url in CSS to the converted output. AFAIK [ext] cannot be overwritten, that’s why I started thinking on custom placeholders, because in my case my own [newext] placeholder could solve this problem nicely.

With loaderContext.emitFile I was able to emit the converted file with proper extension, but I was no able to change the url in CSS with this.callback(null, 'newfilename'). When I avoid emitFile and return the buffer with the converted file in the async callback then the url in CSS is OK, but the extension remains the old one.

How should this be implemented in your opinion?

Maybe some new function to loaderContext / this like setCustomPlaceholder(name, value), but it’s just a sudden idea.

Are you willing to work on this yourself?

Unfortunately I’m not at that level…

[alexander-akait]

I assume that normally resource and resourcePath are referencing to some existing source file - which initially triggered the loader - and renaming/overwriting them would make them not existing (e.g. for upper loaders in the chain). Is that ok? Would not this bring in new problems?

We have resolved path to real file and currently resourcePath used only for code generation, so if you change something in resourcePath, it will change only generated code, anyway there is bottleneck - logic in non official loaders can be different and can be broken. So we need something new in webpack to avoid it, maybe this.virtualResource and webpack will use it for code generation instead of resource/resourcePath and other (just idea).

This all is not in issue for image optimization loaders, because extension and mimetype won’t change after optimization. But if a loader returns a Buffer with new image type, the [ext] placeholder and the initial mimetype gets invalid for filename and base64 dataUrl generation for Asset Modules. This is the real problem and unfortunately a complex one.

You need change mimetype in your loader too, I want to say it is expected, if you change something, you need change not only buffer, webpack doesn’t know about mimetype nothing, now you can do it only using https://github.com/webpack-contrib/image-minimizer-webpack-plugin/blob/master/src/loader.js#L155, but it is dirty workaround (try it)

I know that the loader could save the converted file with this.emit with any name, but in this case Rule.generator.emit has to be disabled and the there’s still the problem of renaming the resource for correct Asset Module filename generation… (e.g. to let the background: url() in CSS to be actualized to the manually emitted file)

It is wrong usage, this.emit adds asset to module, but there is module as asset, I want to say it is wrong direction, for example JS/CSS/HTML can contains assets, so module has assets, but image.png can’t contains assets, because it is already asset… You should avoid using this.emit, because it will not work for code generation (as you write it here), this problem related to initial problem - ability to change module resource.

Anyway I see we have very complex and unpredictable logic, maybe we should move ahread and do some things better:

1. Introduce the new property - resourceURL, store only URLs there (i.e. regular paths are converting to file:///, Node.js allows to use it in most of std API, data/file:/http:/etc stored as is), i.e. we will store new URL(...) result there, to avoid performance problem we can use getter, JS ecosystem is moving to using URLs, so I think we should do the same
2. Allow to change any part of resourceURL and it will be easy, because we have result of the new URL(...) parsing
3. webpack uses resourceURL for code generation, if it was provided by module (i.e. loader sets it)

So we solved two problems - standardized and simplify logic using resourceURL and allow to rename module (also no virtualResource).

Also not fake virtual modules and hacks for them, if you want to have virtual modules, you need to pass/rename module to data: URI and it will be like virtual module (we have requests on this feature).

[alexander-akait]

Do you still want me to open one or are you willing to do that?

Let’s keep open this issues, because we have context of the problem here and ideas how we can fix it, this will allow you not to lose something. Yes, I hope we will improve API in near future, because we need this for official solutions too.

Hopefully, the deprecated this._module won’t be removed sooner.

this._module will be never removed due compatibility problems between versions, so you can use it when you don’t have other solutions (anyway I prefer say to do not use if you have good API)