Cache a node: GL.Node saveBuffer+cached props

We should be able to cache a node (which means part of the rendering tree) using a previously saved buffer. We should also be able to pipe the previous framebuffer as a texture for more advanced use-cases like cellular automaton or simply persistence effects (and making gl-react more compatible with shaders from shadertoy.com and glslsandbox.com). This can also be used in combination of discard in GLSL to re-render only part of a buffer (preserve part of the previous buffer) to allow interesting things like drawing a brush (see https://github.com/ProjectSeptemberInc/gl-react-native/issues/19).

API Draft

• <GL.Node shouldUpdate={false} saveBuffer="foo">: a way to cache node’s render computation.
  • saveBuffer: keep the rendering content for the next frame (using a framebuffer). It’s a bit like ‘key’ semantic in React but it’s global across the whole tree.
  • shouldUpdate: if set to true (default) will make the GL.Node render normally. if set to false, will shortcut the rendering to re-use the previous framebuffer instead.
• special symbols that can be passed to uniforms:
  • GL.PreviousBuffer: the previous rendered buffer.
  • GL.HasPreviousBuffer: boolean to know if GL.PreviousBuffer is provided (if false, PreviousBuffer will be rendered black).

Implementation

gl-react-native and gl-react-dom implementations will have to use a new framebuffer collection that is preserved after each render. This new collection of framebuffers will be solved by gl-react. There is 2 categories: “shared” fbos (if supported by the implementation (e.g iOS), fbos are shared between the surface instances) and “preserved” fbos that exclusively belong to each surface instance. The preserved fbos are identified by an id (user provided in saveBuffer props). (so preservedFbos will be a Map / Dictionary / JSObject from that id to the fbo object). We need a new internal format to describe a fbo: { type: "shared", id: 1 } & { type: "preserved", id: "foo" }. Everywhere we used a fboId, it should adopt this new format (e.g in texture types).

in draw()

in draw() method of implementations we need to do this:

• save glcanvas’ preservedFbos into previousPreservedFbos.
• create a newPreservedFbos.
• every time a uniform references a preservedFbos, we pick it from the previousPreservedFbos map.
• every time a node uses saveBuffer, we still use the shared fboId to store the rendering, but at the end of that rendering, we store that shared fboId into newPreservedFbos and create a new fbo in the shared fboId (this is the safest, but we should try to limit this recreation…, i’m already thinking of a smarter way but that might be premature optim). – I need to rethink that, i’m not 100% sure if this works and if it’s the simpler way
• set newPreservedFbos into glcanvas’ preservedFbos for next draw().

gl-react

there must be an invariant that check there are no dups in saveBuffer in the tree.

This feature would have to be carefully used because storing part of the tree cost one more framebuffer to keep in memory (for each view instance) but in some cases we might want to cache a complex rendering that rarely changes. It’s a tradeoff between GPU computation and Memory.

---

`<GL.Node shader={…} saveBuffer=“foo” cached={true || false} />

• saveBuffer because I think we need a unique id to locate the node if tree changes a lot. this also makes it consistent with #25
• cached is a boolean to indicate the lib should use a previously saved buffer.

---

also, saved buffer can be used in uniforms:

<!--
<GL.Node shader={...}
  uniforms={{
    backbuffer: GL.fromBuffer("foo")
  }}
/>
-->

GL.fromBuffer references a buffer saved with saveBuffer prop in the last rendering frame, it can be the node itself (allows to do persistence effect – in that case, implementation will need to use a trick either to switch between 2 fbos or copy fbos one into another).

[morukutsu]

Hi, I was interested by the “previous frame feature” described in the first post (to implement motion blur / accumulation like effects). Using Surface.captureFrame was too slow for real time as expected.

I implemented a quick demo of the feature here: https://github.com/morukutsu/gl-react-dom/commit/4b6705eeedfec67585d4eda35b40231fef7e52b4

There are still some issues:

• implementation is hacky because of the magic number I use for the previous frame fbo id
• previousFrame is a hardcoded name which triggers the feature, using the “GL.PreviousBuffer” semantic would be better
• I didn’t check the implementation for the native version
• The previous frame works at the Surface level. Ideally, it would be nice to be able to retrieve the previous frame of any GL Node.

What are your plans regarding this feature? I guess it is not really useful for most use cases but I kinda like it! And it’s not too hard to implement properly.

[gre]

Sorry to have been a bit too talkative in my 5-days-ago comment, but basically most issues I was talking about are fixed in a PoC implementation I’ve started this weekend ( see also https://github.com/ProjectSeptemberInc/gl-react/issues/70#issuecomment-254172991 ). Still a WIP, I’m exploring different things and still change the API a lot, so I won’t publish this right now, but soon I hope 😉 I will also have to see how I can port this to native, and how I can reuse the maximum code on JS side (which also might change some API).

Basically I do what you said: less magic in the lib and more control to user, in short: dropping the dedupe logic, embracing React lifecycle.

I’m using the context to connect things together.

One “hack” is that GL.Node render() method is like: <span>{all uniform sub nodes}</span>, so it’s rendered normally in React (and i can use lifecycle!! important part). I also even put that inside the <canvas> tag (it can have content, it’s just not visible. well, perfect!!)

I attach one framebuffer to each GL.Node instance. (except for the “back buffering” usecase, you just set a backbuffering prop, and you have 2 fbos that get swaped each draw time)

I have not yet thought about sharing and reusing fbos, attaching one fbo per GL.Node makes things so much simpler and allow to do things like caching (redraw only if changes, otherwise keep the fbo in memory) as well as allowing to snapshot a specific GL.Node without need to redraw it (just takes what’s in the fbo). But still, I know there might be some usecases where you want to make fbos more global (i’m thinking about blur with lot of passes OR in native case, when you have lot of concurrent surfaces). I was considering a framebufferGroup concept, where you would set a number to make a sharable fbo across many nodes… anyway, that’s complexifying the implementation, so i’ll see later.