Updating textures in VR causes weird objects to appear

Description of the problem

I’m working on a project like MS Paint in 3D using Three.js and WebVR.

I’m trying to draw on texture using HTML canvas and canvasTexture. I’m just using lines and arcs on a virtual canvas at the place of uvs given from raycast with plane intersection. Everything here works ok.

The problem:

When I use any “handheld” drawing tool (e.g. pencil), I need to update the texture every frame to show the drawing in realtime. Then, only in VR (on desktop everything is ok), on my plane mesh (and on some other meshes added to the scene, but only a few) a weird alpha clipping mask appears which covers pixels of my painting/drawing (when transparency is enabled it replace drawing pixels with fully transparent pixels) (see screenshots 3 ,4, 5 and 6). This weird mask moves with me and follows me when I’m moving my head in the VR headset, it is not stationary (see screenshot 2). The appearance of the mask seems to be a set of objects in front of the camera when I’m entering VR mode (see screenshot 0).

But, when I stop drawing/painting and stop updating the texture, everything is ok and the drawing/painting displays normally (see screenshot 1).

After some investigation I figured out that the problem is in only one line of code, which sets texture needsUpdate boolean to true.

I cannnot share whole code or application, but I’m attaching some pieces of it.

Screenshot 0

The first render of scene after entering VR mode.

Screenshot 1

Normal display of drawing canvas when there is no need to update texture. There is transparency on canvas because of (only for testing and having a possibility to show the problem better) disabling material transparency.

Screenshot 2

Weird mask shows on loaded mesh and on background behind the window.

Screenshot 3

Mask shows on drawing palette mesh.

Screenshot 4, 5, 6

Shows the mask and how it follows head movement.

Pieces of class #1 which creates meshes and materials.

        const canvasGeometry = new THREE.PlaneBufferGeometry(this.canvasDimensions.w, this.canvasDimensions.h);
        this.canvas = new THREE.Mesh(canvasGeometry,
            new THREE.MeshBasicMaterial({side: THREE.FrontSide, transparent: false}) //normally transparent value is true, but false shows how "mask" looks like
        );
        this.canvas.position.x = this.canvasDimensions.w / 2;
        this.canvas.name = 'drawing palette';
        this.add(this.canvas);

        this.canvasBackground = new THREE.Mesh(canvasGeometry,
            new THREE.MeshBasicMaterial({side: THREE.FrontSide})
        );
        this.canvasBackground.position.x = this.canvasDimensions.w / 2;
        this.canvasBackground.name = 'drawing palette background';
        this.add(this.canvasBackground);

Pieces of class #2 which handles drawing with HTML canvas part.

       this._canvas = document.createElement('canvas');
       this._canvasContext = this._canvas.getContext('2d');
   ///	
       this.canvasTexture = new THREE.Texture(this._canvas);
       this.canvasTexture.needsUpdate = true;
       this.paletteMesh.material.map = this.canvasTexture;
       this.paletteMesh.material.needsUpdate = true;
   ///
   	drawHand() {
   		this._canvasContext.beginPath();
   		if (!this.buttonDownThisFrame) {
   			this._canvasContext.moveTo(this.previousFrameDrawPosition.x, this.previousFrameDrawPosition.y);
   		}
   		this._canvasContext.lineTo(this.thisFrameDrawPosition.x, this.thisFrameDrawPosition.y);
   		this._canvasContext.stroke();
   		this._canvasContext.arc(this.thisFrameDrawPosition.x, this.thisFrameDrawPosition.y, this.brush.getSize() / 2, 0, 2 * Math.PI);
   		this._canvasContext.fill();
   		this._canvasContext.closePath();

   		this.canvasTexture.needsUpdate = true; // when this flag is set to true (when button is down) "mask" appears
   	} 

Three.js version

• Dev
• r94 (and previous)
• …

Browser

• All of them
• Chrome
• Firefox
• Internet Explorer
• Occulus browser

OS

• All of them
• Windows
• macOS
• Linux
• Android
• iOS
• Samsung Gear
• Occulus GO

Hardware Requirements (graphics card, VR Device, …)

• Samsung Gear
• Occulus GO

[Mugen87]

I’ve debugged your app but I could not find the problem. A few things I’ve noticed:

You have statements in your code which are not necessary or wrong. I don’t understand why you set…

• WebGLRenderer.sortObjects to false,
• WebGLRenderer.localClippingEnabled to true,
• the alpha parameter of WebGLRenderer to true
• the preserveDrawingBuffer parameter of WebGLRenderer to true

Besides, I’ve tested with Google Daydream and the 3D effect looks wrong (like the left and right eye are not in sync). The official three.js examples don’t have this visual problem https://threejs.org/examples/?q=webvr. BTW: Both planes are black and I’m not able to draw anything.

One problem is that your provided example is still too complex. You have to provide a more simple test case without any node modules and 3rd party libraries. Just use the classes of thethree.js repo.

In any event, as long as you can’t demonstrate a bug I’m inclined to close this issue.

[mrdoob]

I also have a bug with Oculus Browser (6.2.11.181027543) on Oculus Go. The sentence “Updating textures in VR causes weird objects to appear”, exactly describe my problem. The issue doesn’t seems to happens on Gear VR.

You should probably report the bug to Oculus.