`fromPixels` returns invalid values for `webgpu` backend

i’ve noticed some errors during model execution, so went back to check inputs and it seems that fromPixels returns incorrect values for webgpu backend

await tf.setBackend('webgpu');
// input is a canvas which is a copy of a webcam input in 1280x720 resolution
const buffer = await tf.browser.fromPixelsAsync(canvas);
const data = await buffer.data();
for (let idx = 0; idx < data.length; idx++) {
  if (data[idx] < 0 || data[idx] > 255) console.log({ idx, val: data[idx], length: data.length });
}

{idx: 1094922, val: -1, length: 2764800}
{idx: 1094925, val: -1, length: 2764800}
{idx: 2275694, val: -1, length: 2764800}

i cannot think of any scenario where pixel value of -1 is a valid thing to have,
especially on only 3 random pixels out of entire (otherwise valid) image
note that invalid values do not occur on every frame, but it does happen often enough

processing the same input shows no such problems when using webgl backend

environment: tfjs 3.20.0, tfjs-backend-webgpu 0.0.1-alpha.13, chrome canary 107

[qjia7]

@shaoboyan Please file a chromium/dawn bug to trace the GPUExternalTexture issue. And this issue will be blocked by that one. @vladmandic To continue to use fromPixels with video, you can set flag WEBGPU_IMPORT_EXTERNAL_TEXTURE to false. In this path, it will package the video to canvas and then call copyExternalImageToTexture() to do the following thing.

[vladmandic]

@shaoboyan live link i’ve posted (https://vladmandic.github.io/nudenet/src/verify.html, sorry there was a typo in the url) includes a sample video. its only 10sec long, but on my system it generates thousands of errors per frame - and it logs to browser console.

example:

{tf: '3.20.0', backend: 'webgpu', available: {…}, flags: {…}}
{mediaTime: 0.64, errors: 44492, videoData: Int32Array(8294400), canvasData: Int32Array(8294400)}