Make shaders tree-shakable (inspired by threeify's approach)

Is your feature request related to a problem? Please describe.

Currently the shader chunks are taking up a significant amount of space in the Three.js final bundle. 126KB of 666KB - so roughly 20% of the space.

This is clear from this image shared on Twitter: https://twitter.com/threejs_org/status/1382639620119216131

This is the first of two feature requests on how to reduce further Three.j’s build sizes based on my learnings from threeify. The other is here: https://github.com/mrdoob/three.js/issues/21667

Describe the solution you’d like

To make the shader chunks tree shakable, one can take inspiration from how threeify did this. Threeify can have build sizes as small as 12KB compressed JS. https://threeify.org/examples/brdf_clear_coat_specular

The way it did this is:

• The glsl files in the project are transpiled to *.js files where the “#include <file>” statements are replaced with JavaScript import directives. This happens automatically behind the scenes.
• When using *.glsl files you can import then directly as you would any other module. They are actually *.js files but because many include systems can just auto append *.js, you can include them as *.glsl. Thus your code looks like this: https://github.com/threeify/threeify/blob/master/src//examples/brdfs/clearcoat/index.ts https://github.com/threeify/threeify/tree/master/src/examples/brdfs/clearcoat
• I wrote this transpiler to do this live *.glsl to *.js translation: https://github.com/threeify/threeify-glsl-transpiler
• Because everything is based on JavaScript imports, everything is tree shaken just to the code you are using.
• There is no need for a run-time include system for the GLSL files because they just piggy back on the JS include system.
• The glsl-to-JS transpiler I wrote even has options to minimize the produced GLSL files so that you can remove debug information. This can further reduce the resulting bundle size when you just want production code.

Describe alternatives you’ve considered

I would suggest the above.

Additional context

• There is no dependent upon typescript for this solution.
• It is likely the glsl-transpiler can be directly used in three.js for this purpose. Maybe you could do it inline so that the *.glsl.js files are directly beside the source *.glsl files?
• If the build/server system doesn’t support *.glsl completion to *.glsl.js, you can just decide to import *.glsl.js files directly if needed. A little less clean but all the benefits of this system work the same.

Example of transpile

An original rgbe.glsl file:

#pragma once
#pragma include "../../math/math.glsl"

vec4 rgbeToLinear( in vec4 value ) {
	return vec4( value.rgb * exp2( value.a * 255.0 - 128.0 ), 1.0 );
}

vec4 linearToRGBE( in vec4 value ) {
	float maxComponent = max( max( value.r, value.g ), value.b );
	float fExp = clamp( ceil( log2( maxComponent ) ), -128.0, 127.0 );
	return vec4( value.rgb / exp2( fExp ), ( fExp + 128.0 ) / 255.0 );
}

Will be transformed into a rgbe.glsl.js JavaScript module:

import _math_glsl from "../../math/math.glsl.js";

export default /* glsl */ `
#ifndef _rgbe_glsl // start of include guard
#define _rgbe_glsl

${_math_glsl}

vec4 rgbeToLinear( in vec4 value ) {
	return vec4( value.rgb * exp2( value.a * 255.0 - 128.0 ), 1.0 );
}

vec4 linearToRGBE( in vec4 value ) {
	float maxComponent = max( max( value.r, value.g ), value.b );
	float fExp = clamp( ceil( log2( maxComponent ) ), -128.0, 127.0 );
	return vec4( value.rgb / exp2( fExp ), ( fExp + 128.0 ) / 255.0 );
}

#endif // end of include guard
`;

Advanced

• It is possible to do this type of include system on a function level rather than just on a file level. That is more advanced by it is similar to how you can import just a few functions from a JS module rather than everything. This could be extended to glsl with more advanced transpilation options. But that can be done in the future.

[marcofugaro]

This seems tree-shakable no?

It’s not, I have tested this, there is the mergeUniforms() which modifies the object literal, so it’s not pure anymore.

---

I think that we could modify BasicMaterial to be something like:

import meshbasic_vert from 'shaders/meshbascic_vert.glsl';
import meshbasic_frag from 'shaders/meshbasic_frag.glsl';

export class BasicMaterial {

  vertexShader = meshbasic_vert;
  fragmentShader = meshbasic_frag;

   uniforms = mergeUniforms( [
    UniformsLib.common,
    UniformsLib.specularmap,
    UniformsLib.envmap,
    UniformsLib.aomap,
    UniformsLib.lightmap,
    UniformsLib.fog
  ] );

  // ...

};

I agree, this is the way to go to enable tree-shaking. ShaderLib needs to go, users will be able to access the shaders/uniforms from the class. Maybe we could even make them static.

However we can still leave ShaderChunk (converting it to modules), as long as it’s not used directly. It can be still exported like MathUtils without breaking the API.

https://github.com/mrdoob/three.js/blob/52ec6876d693670629f9618ba7d37f05be5de227/src/Three.js#L109

[raphaelameaume]

I agree, we could still have an index.js file at the root of the shader folder exporting all the shaders so we can do

import * as ShaderLib from "path/to/all/shaders/index.js"

I think we should be able to do so in the library and it would still be tree-shakable:

import { meshbasic_vert, meshphysical_vert, ... } from "path/to/all/shaders/index.js";