WebGL2: Asynchronous Readback with PIXEL_PACK_BUFFERs

Is your feature request related to a problem? Please describe.

WebGL2 (which is now supported by all modern browsers) offers powerful General Purpose GPU capabilities. three.js already implements an impressive array of these features (like WebGLMultipleRenderTargets). This GPGPU suite is nearly complete, but it’s missing a performant way of reading texture data back from the GPU. renderer.readRenderTargetPixels() waits for a sync with the GPU before the download starts, costing tens of milliseconds per-call on PC and more on mobile.

Describe the solution you’d like

WebGL2 adds the ability to bind _gl.PIXEL_PACK_BUFFERs, which allow for asynchronous readback from the GPU Texture Buffers, largely eliminating the performance impact of GPU -> Code. This should open up all kinds of GPU Physics (and other GPGPU) Solutions.

Describe alternatives you’ve considered

Luma.gl implemented async readback a few years ago Babylon.js also has some functionality for this (One implementation in Babylon…) Unity has a similar mechanism for this

Additional context

My immediate use-case would be to speed up this GPU Finite Element Modelling simulation. If I can get everything running fast a stable on all platforms, I’d also like to contribute a new GPUComputationRenderer (which would support WebGLMultipleRenderTargets and separate the concept of “passes” from “variables”, allowing for multiple passes over the same data).

[zalo]

@snsie Thank you for the tips! Following your instructions, I was able to get Async Readback working in my TetSim with these changes: https://github.com/zalo/TetSim/commit/9696c2e1cd6354fb9bd40dbd299c58f4de0341dd

Live Version Here

It results in a dramatic performance improvement when reading texture data back from the GPU! (Especially on Mobile/iOS!)

I didn’t try too many things, but the spooky(?) thing about it is that it only seems to work if it’s called right after the last render() call that writes to the texture you want (I guess because then it’s the active render target?) It may also be desirable to persist buf between calls so gl.createBuffer() isn’t getting called all the time.

[Mugen87]

I can only say that it’s best to start with something simple that can be enhanced over time. Try to not over-engineer the feature. The advantage of PIXEL_PACK_BUFFER should become clear in a single use case which is easy to understand. It’s then more likely that a PR gets properly reviewed and eventually merged.