LUTEffect produces incorrect results

Description of the bug

The newly added LUTEffect currently produces inaccurate results. With a neutral LUT, the image becomes darker even when using HalfFloatType buffers and a highp LUT sampler with FloatType data. Upscaling with the TetrahedralUpscaler via LookupTexture3D.scaleUp() produces banding artifacts.

To Reproduce

The following example uses a neutral LUT which incorrectly darkens the image: https://codesandbox.io/s/fervent-kapitsa-3jikk

When using a 2x2x2 LUT, the image quality is greatly reduced which does not happen here: https://threejsfundamentals.org/threejs/lessons/threejs-post-processing-3dlut.html

Expected behavior

An identity LUT should not change the image. Color space should also not matter if a neutral LUT is used, but linear input colors currently produce even worse results.

Screenshots

Images taken from https://github.com/vanruesc/postprocessing/issues/231#issuecomment-753638346:

Neutral LUT 32	Neutral LUT scaled up to 128

Library versions used

• Three: 0.124.0
• Post Processing: 6.19.0

Desktop

• Windows 10
• Firefox 84.0.1
• NVIDIA GTX 1060

[vanruesc]

I’ve investigated this further and ended up implementing tetrahedral interpolation and custom input domains. I found out that the noisy errors we’re still seeing are indeed caused by hardware interpolation. This article by iq explains the details.

The tetrahedral interpolation algorithm performs all interpolations manually to avoid the quantization of fractional components. The following comparison shows the difference between the input colors and the output colors of the LUTEffect using a neutral 2³ LUT:

HW Trilinear 2D	HW Trilinear 3D	Manual Tetrahedral 3D

Tetrahedral interpolation eliminates all remaining errors.

@gkjohnson FYI: I learned that fract() actually uses floor() internally: “This is calculated as x - floor(x).” So the following is probably a bit more efficient for the 2D sampling algorithm:

float slice0 = floor(slice);
float interp = slice - slice0;

[gkjohnson]

Good to know that looks like a useful site!

If the visual result looks good I think I’m content to leave it for now. I won’t rule out the sampling function still having some issues but with more calculations happening in the 2d sampling case I’d imagine you’d get some differences from floating rounding error, as well, so aiming for literally identical images may be an impossible goal.

And after taking a quick look at the TetrahedralUpscaler again I don’t think the sampling approach needs to be adjusted as I suggested in https://github.com/vanruesc/postprocessing/issues/250#issuecomment-755074282 and the visual result looks okay so I think that component is good.