ImmersiveControls

Making a good immersive VR controls module is long and complex, because of the many basic cases to cover :

• no immersion ( mouse/keyboard preview before immersion )
• immersion but no controller
• one controller ( right/left )
• two controllers
• hands controls

This is an issue because :

• It might discourage people from going into VR development with three.js.
• It’s a lot of code if you only want to make a quick demo ( for showcases, bug reports, experiments… ).
• Handling events, raycasting and intersections with all the above-mentioned cases in a consistent way is complex.
• Everybody make their own makeshift imperfect solution, which is suboptimal

I found myself creating a “VRControls” module on my first VR project, and re-used it ever since, improving it now and then… and I’m sure everyone using three.js for VR do the same thing ( not suggesting to use mine, it’s unsuitable ). I think it would be beneficial for everybody if we had a community tool for this, a new controls module in the exemples.

Example API :

import { ImmersiveControls } from 'three/examples/jsm/controls/ImmersiveControls.js';

const controls = new ImmersiveControls( camera, domElement );

controls.addEventListener( 'click', (e) => {

	e.origin; // 'mouse', 'right-controller', 'left-hand'...

	// cast a ray either from the camera or the controller(s)
	const intersects = controls.intersectObjects( scene.children );

});

controls.addEventListener( 'move', (e) => {

        // intersect bounding sphere(s) at controller(s) or hand joint(s) position
	if ( controls.intersectsPlane( plane ) ) {

		console.log( `then ${ e.origin } is intersecting the plane` )

	}

});

See how life would be easier : the three.js user would not have to care about what is the current state of immersion. They would just listen to standard events and do intersection tests from a higher-level API. It would make for a more standard and healthy code.

Of course this would not fit every case of every VR project, but most of the time it would be enough, and it would definitely help people hit the ground running.

If there is interest for this I’m willing to work on a PR ( or to let somebody more capable do it 😉 ) after discussing the scope and API.

[mrdoob]

@felixmariotto Sorry for the delay.

It’s hard for me to know how the design for this should be without doing experiments…

So that’s what I’m planning to do for the next few weeks 🤓

[paulmasson]

@mrdoob any recent thoughts on this issue?