Broken surface ID buffer on GTX 1070 ?

Reported on Twitter: https://twitter.com/tamat/status/1580872701035495425

I haven’t been able to reproduce. My guess is the surfaceID buffer has invalid data drawn to it. That buffer is of type HalfFloat. I picked that because I thought that’d have more support than Float buffers, but it may be the other way around?

This buffer type is defined here:

https://github.com/OmarShehata/webgl-outlines/blob/b591487f9ae9c1fe450580be1f3b02cf58dd19d1/threejs-outlines-minimal/src/CustomOutlinePass.js#L28

[dtrebilco]

OK I think I know what is causing this:

1. The object ID is read by the vertex shader (verified correct) and passed as a basic interpolator to the fragment shader.
2. The fragment shader basically just writes this object ID out to the Float16 buffer
3. Post process is applied on these id buffer values where if surfaceValueDiff != 0

As these id values are floating point, any slight variation will produce a outline. (in general, comparing floating point values for exact values causes problems)

There are 3 ways to fix it: Solution 1) Don’t compare against exact values - change to surfaceValueDiff > 0.01 - that seems to work

Solution 2) Make sure no perspective correct interpolation is happening on the varying that will not need it - this ensures you get the exact value in the fragment shader - just put a “flat varying” in front of the shader interpolator when writing object ids. However, this needs WebGL2. FYI: Here is a good reference on perspective correct interpolation if you did not know about it: https://stackoverflow.com/questions/24441631/how-exactly-does-opengl-do-perspectively-correct-linear-interpolation

Solution 3) Another way to prevent perspective correct interpolation is to make sure position “w” value is 1. Simply do a gl_Position.xyz /= gl_Position.w; gl_Position.w = 1.0; in the vertex shader writing vertex ids.

Side note: I think you need to clamp the value “outline” when doing

gl_FragColor = vec4(mix(sceneColor, outlineColor, outline));

as outline could be > 1.0

Just adding a note here how I debugged this in case I ever need to revist: Using this video as a starting point https://www.youtube.com/watch?v=T7MPxvX5alA Using RenderDoc on an old version of ChromePortable ChromiumPortable.exe --no-sandbox --disable-gpu-watchdog --gpu-startup-dialog --disable-direct-composition Attached RenderDoc by injecting into process (this option is enabled via a toggle in settings)

Also tried to use Chrome canvas debugger, but it is removed from modern chrome. Some initial testing with https://spector.babylonjs.com/ plugin, but this does not support shader editing.

[dtrebilco]

Verified live demo looks correct for me