WebGL renderer?
See original GitHub issueEDIT: Tracking progress:
- Bitmap + vector sprites
- Speech bubbles
- Pen layer
- Sprite is touching mouse
- Sprite is touching sprite
- Effects
- Layer ordering
- Rotation style
- Convex hull generation / “if on edge, bounce”
- Sprite is touching color
- Color is touching color
- Resizable stage
- Subpixel positioning of SVG viewboxes
Incorporating WebGL in some form is necessary to implement graphic effects at a reasonable speed.
I was thinking of rewriting the renderer to use WebGL at its core, similar to scratch-render. This could potentially take care of a number of currently missing blocks in one fell swoop:
- Effect blocks
- Layering
- Touching color
- If on edge, bounce (requires a “precise” bounding box, and hence convex hull generation, for Scratch-compatible results)
There are a couple of implementation details I’m not sure about, though:
In regular Scratch, each sprite has a fixed number of costumes, and you can only create costumes by clicking a button in the editor. In scratch-js, on the other hand, you can dynamically instantiate as many Costume
objects as you like.
This presents a problem because WebGL textures are never garbage-collected. In Scratch, the textures that back a costume are deleted when the costume is. In scratch-js, though, you could (as far as I can tell) do something silly like create and render a new Costume
object every frame.
I can think of three ways to deal with this:
- Add a
destroy
method toCostume
s that you require API consumers to call. - Maintain a list of every created texture in the
Renderer
. Every rendering step, mark every texture that ends up getting rendered, then sweep away those that aren’t. - Give each texture a refcount. At every call site that adds/removes textures (e.g. the
Sprite.costume
setter), update the refcount and delete the texture if the count is 0.
I’m leaning heavily towards the second option myself–it seems like the least complex one.
The other implementation detail that might be affected is pen color, which is also being discussed in #22. While the canvas API accepts any CSS color string, WebGL requires colors in an RGBA format. That may factor into the pen color discussion somewhat, as every accepted pen color format will need to be converted into RGB(A).
I’d love to hear your thoughts on all of this–this is a really promising project!
Issue Analytics
- State:
- Created 4 years ago
- Reactions:4
- Comments:13
Top GitHub Comments
I’d hope not–I’ve already started in on it!
If you look inside
Renderer.mjs
and therenderer
folder, you can get a feel for the complexity WebGL adds. You’re correct that it’s somewhat more difficult to understand than the canvas API, especially if you’ve never seen it before, just because it’s “closer to the metal”, but I think that it’s a fair tradeoff given how much more you can do with it.I’ve tried to abstract away the more boilerplate-y stuff (in classes like
ShaderManager
) and I’ll probably clean it up more as I progress. I also haven’t yet heavily commented the code.If this is your first time looking at WebGL code, I’d recommend looking at WebGL Fundamentals or the MDN tutorial to get a feel for how the API functions.
The reasons I don’t want to go with Pixi are:
scratch-js
will never use.scratch-js
Sprite
s toPIXI.Sprite
s, and managing the creation and deletion ofPIXI.Texture
s (which, to be fair, you also need to do in WebGL).I considered
Map
s too, but they can’t be reordered which kinda defeats the purpose of using them for layering.