Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Posenet estimation requires explicit setting of the `width` and `height` properties on the input object.

See original GitHub issue

TensorFlow.js version

"@tensorflow-models/posenet": "^0.0.1" "@tensorflow/tfjs": "0.10.3"

Browser version

Google Chrome | 66.0.3359.181 (Official Build) (64-bit)
-- | --
Revision | a10b9cedb40738cb152f8148ddab4891df876959-refs/branch-heads/3359@{#828}
OS | Mac OS X
JavaScript | V8 6.6.346.32

Describe the problem or feature request

When trying to implement the Posenet model, I have encountered an interesting problem. Passing an HTMLVideoElement into estimateSinglePose without it explicitly having a width and height property set results in this error:

Uncaught (in promise) Error: Requested texture size [0x0] is invalid.
    at Object.validateTextureSize (es6.promise.js:287)
    at createAndConfigureTexture (es6.promise.js:287)
    at Object.createMatrixTexture (es6.promise.js:287)
    at GPGPUContext.createMatrixTexture (es6.promise.js:287)
    at TextureManager.acquireTexture (es6.promise.js:287)
    at MathBackendWebGL.uploadToGPU (util.js:264)
    at MathBackendWebGL.getTexture (promise.js:8)
    at MathBackendWebGL.fromPixels (es7.promise.finally.js:21)
    at Engine.fromPixels (_perform.js:8)
    at ArrayOps.fromPixels (_uid.js:6)

I suspect this API uses tf.fromPixels somewhere underneath (the signature looks eerily similar) and that API expects to read the width and height from the object itself.

My question would be is this intended? Should this be documented somewhere (unless it is already and I have missed it)? Should the type interface force those properties to be set somehow?

Code to reproduce the bug / link to feature request

<html>
<body>
  <script src="./index.js"></script>
  <div>
    <video id="video-stream" style="width: 600px; height: 600px;">
  </div>
</body>
</html>

  const net = await Posenet.load(1.01);

  const mediaStream = await navigator.mediaDevices.getUserMedia({
    audio: false,
    video: {
      height: 600,
      width: 600,
      facingMode: 'user',
    }
  });

  const videoElement = document.getElementById('video-stream');

  // If the below two lines are missed out, the error appears!
  // videoElement.width = 600;
  // videoElement.height = 600;

  videoElement.srcObject = mediaStream;
  videoElement = await new Promise((resolve, reject) => {
    videoElement.onloadedmetadata = () => resolve(videoElement);
  });
  videoElement.play();

  // The next line throws if the object does not have `width` and `height` explicitly set!
  const pose = await net.estimateSinglePose(videoElement);

Issue Analytics

State:
Created 5 years ago
Reactions:2
Comments:20 (8 by maintainers)

Top GitHub Comments

4reactions

kylemcdonaldcommented, Feb 25, 2019

I’m currently seeing “Error: Requested texture size [0x0] is invalid.” which lead me to this page. I suggest an error like “Input object width and height properties are not set.”

To fix my case I’m using the suggestion above:

videoElement.width = videoElement.videoWidth;
videoElement.height = videoElement.videoHeight;

This also suggests that tfjs might possibly use videoWidth/videoHeight instead of width/height when processing video elements?

3reactions

oveddancommented, May 23, 2018

@FenrirWillow thanks for the detailed explanation. You need to give the element an explicit width and height via attributes. Otherwise, Tensorflow.js, and PoseNet, would have no way of knowing what the width and height of the element are, without doing something like getBoundingClientRect. PoseNet needs this to properly scale the image before feeding it through the network.

This should be documented, or we should throw an error if the width and height of the element is not known.