Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Initially muted video breaks videoTexture

See original GitHub issue

I am doing some experiments using webcam input and saving processed video with audio to a local file. I have encountered a strange problem, that may be a browser bug. I am using one of the most recent versions of Chromium for Windows. Below is a simple script (with no image processing) to reproduce the problem. The record button starts and stops a recording. Without muting the audio, there will be feedback. Therefore I want to mute the audio for the video element that is being used as a texture, but keep the audio for the recording.

Setting muted:true during the Object.assign (remove // from //muted: true, in the script below) causes this error message: WebGL: INVALID_VALUE: tex(Sub)Image2D: video visible size is empty and afterwards setting mute to false does not help.

Strangely, letting muted be false initially, it can be toggled using dat.GUI and achieve the desired effect.

I can work around the problem by creating a new pure video stream for the video element, but it would be better to understand why muted:true does not work.

<!doctype html>

<html>
<head>

<script src="http://threejs.org/build/three.js"></script>
<script src="http://threejs.org/examples/js/libs/dat.gui.min.js"></script>

</head>
<body>
<script>
var videoTexture, videoSettings, videoStream, audioTrack;

let gui = new dat.GUI();

navigator.mediaDevices.getUserMedia({video: true, audio: true}).then(function(stream) {
	videoSettings = stream.getVideoTracks()[0].getSettings();
	audioTrack = stream.getAudioTracks()[0];
	//Making a separate pure video stream is a workaround
	//let videoStream = new MediaStream(stream.getVideoTracks());
	let video = document.createElement("video");
	Object.assign(video, {
		srcObject: stream,//videoStream,
		autoplay: true,
		//Setting muted here breaks everything
		//muted: true,
	});
	//Toggling muted later works as expected, if muted was initially false:
	gui.add(video, "muted");
	//document.body.appendChild(video);
	videoTexture = new THREE.VideoTexture(video);
	videoTexture.minFilter = THREE.LinearFilter;
	init();
	}
	).catch(function(error){console.error(error);});

var renderer, scene, camera;

function init() {
	let w = videoSettings.width;
	let h = videoSettings.height;

	//Renderer setup
	document.body.style = "overflow: hidden;";
	var container = document.createElement("div");
	document.body.appendChild(container);
	renderer = new THREE.WebGLRenderer({antialias: true});
	renderer.setSize(w, h);
	container.appendChild(renderer.domElement);
	
	//Scene setup:
	scene = new THREE.Scene();
	
	let display = new THREE.Mesh(
		new THREE.PlaneBufferGeometry(2, 2),
		new THREE.MeshBasicMaterial({map: videoTexture})
	);
	scene.add(display);
	
	//Camera setup:
	camera = new THREE.OrthographicCamera(-1,1,1,-1);
	camera.position.z = 1;
	//scene.add(camera);

	videoStream = renderer.domElement.captureStream(videoSettings.frameRate);
	videoStream.addTrack(audioTrack);
	
	let data, mediaRecorder;
	let recording = false;
	gui.add({record: function() {
		if (!recording) {
			data = [];
			mediaRecorder = new MediaRecorder(videoStream);
			mediaRecorder.ondataavailable = function(e) {data.push(e.data);};
			mediaRecorder.onstop = function(e) {
				let blob = new Blob(data, {type: "video/webm"});
				let a = document.createElement("a");
				a.download = "Video_recording.webm";
				a.href = window.URL.createObjectURL(blob);
				a.click();
			}
			mediaRecorder.start();
			recording = true;
		} else {
			mediaRecorder.stop();
			recording = false;
		}	
	}},"record");
	
	
	setInterval(function() {
			renderer.render(scene, camera);
		}, 1000./videoSettings.frameRate);
}
</script>
</body>
</html>

Issue Analytics

State:
Created 5 years ago
Comments:11 (6 by maintainers)

Top GitHub Comments

7reactions

EliasHaslecommented, Oct 24, 2018

I confirmed the bug in pure WebGL, and filed a report here: https://bugs.chromium.org/p/chromium/issues/detail?id=898550

@WestLangley It wasn’t exactly a help request (I found and documented workarounds), and @Mugen87 for all I knew at the time it could have been a three.js bug. I have spent a lot of time on my own replicating the bug in WebGL. Before that, there could be a chance one of you knew something. I guess, had I submitted a bug report to the Chromium team using only three.js code, they could have closed the bug and told me to go to you first. But thanks to both of you for responding to my report. 😃

4reactions

jufacommented, Jan 16, 2020

Followup in case anyone finds their way here: setting the video playhead to > 0 seemed to have resolved this in my application using an mp4 in a VideoTexture instance, i.e. video.currentTime = 1;