SelectionBox: Add support for pixel-perfect selection.

P.S. it is not question, it is feature request for frustum.intersectsGeometry(mesh.geometry)

I mean not bounding box and bounding sphere, but exactly pixel perfect, I mean.

Maybe shader or special renderer? Give different colors to different meshes, and by rendered colors detect what objects were selected?

After three days of googling, I haven’t found a single answer on the Internet.

For example, this Box should not be selected:

[gkjohnson]

In terms of alternatives in other projects something similar (lasso selection of individual triangles) was requested a bit ago in three-mesh-bvh and I’ve created an example for it here. It uses the bounds tree to accelerate selection but you can disable it to see the difference in performance (ms displayed in bottom left). The core the approach involves projecting the triangles to screen space and performing the line-crossing algorithm to determine triangle intersection so you don’t need an acceleration structure to implement it. If you’re limiting selection to a box you can use the projected frustum shape and check every triangle, instead, which might be a bit faster / simpler.

Anyway it might be at least a start for someone looking to implement this in a project. Other relevant discussions in three-mesh-bvh issues 109 and 166.

[gkjohnson]

Maybe we can paint objects in the scene in different colors, and render the scene, and then use the pixel colors to understand what colors are visible? And so we exploit the parallel nature of the graphics card? Its like raycasting, but by every selection pixel.

This would make selection asynchronous which might not be desirable. Copying render target data from the GPU to to the CPU can also be a slow operation and will scale based on canvas size / pixel density. GPU picking for raycasting is a bit more consistent because you only need to render / copy a single pixel.

How does Blender do it? If I select many objects, he clearly understands what I have selected. Do you have any ideas about their algorithm?

Blender may have extra underlying acceleration data structures to facilitate performant interactions like this that three.js doesn’t provide out of the box. With simple geometry performance may not be all that bad though. For all the meshes you want to check selection for you could perform a triangle / selectionbox frustum intersection for every triangle in the mesh to see if it is selected. If the bounding box / sphere of a mesh is completely contained in the frustum you could skip the triangle check and just mark it as selected. Supporting all the corner cases of instanced, skinned, etc meshes may get complicated, though.

If performance turns out to be an issue you could only run the selection check when the user releases the mouse rather than on every frame. For particularly complex meshes you could use some kind of spatially indexed data structure like a quad tree or bvh which there are some libraries for.