tfjs-models: make pose-detection deno compatible

I am attempting to get tensorflowjs running within deno. A basic script like the following will work:

import * as tf from 'https://cdn.skypack.dev/@tensorflow/tfjs'
import 'https://cdn.skypack.dev/@tensorflow/tfjs-backend-webgpu'

// initialize tensorflow
await tf.setBackend('webgpu')
await tf.ready()
const tensor = tf.tensor1d([0])
tensor.print()

deno run --allow-write --allow-read --unstable mod.ts

I can also load up a model, specifically over http, using the file:// protocol actually still triggers a fetch, which deno fails on.

import * as tf from 'https://cdn.skypack.dev/@tensorflow/tfjs'
import 'https://cdn.skypack.dev/@tensorflow/tfjs-backend-webgpu'

// initialize tensorflow
await tf.setBackend('webgpu')
await tf.ready()

const model = await tf.loadGraphModel(`https://storage.googleapis.com/tfhub-tfjs-modules/google/tfjs-model/movenet/singlepose/lightning/4/model.json`, {
  fetchFunc: fetch
})

However, I would like to be able to reuse some of the code in tfjs-models, since there is a decent amount of code that makes the output from the models useful. This is the following code snippet I want to get working:

import * as tf from 'https://cdn.skypack.dev/@tensorflow/tfjs'
import 'https://cdn.skypack.dev/@tensorflow/tfjs-backend-webgpu'
import * as poseDetection from 'https://cdn.skypack.dev/@tensorflow-models/pose-detection';

// initialize tensorflow
await tf.setBackend('webgpu')
await tf.ready()

const detectorConfig = {
  modelType: poseDetection.movenet.modelType.SINGLEPOSE_LIGHTNING,
}
const detector = await poseDetection.createDetector(poseDetection.SupportedModels.MoveNet, detectorConfig)

System information MacOS M1

• TensorFlow.js version (you are using): v3.14.0
• Are you willing to contribute it (Yes/No): Yes

Describe the feature and the current behavior/state. tfjs throws an error

error: Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'fetch')
      this.fetch = env().platform.fetch;

Will this change the current api? How? yes, adding a third option to the createDetector method. Example:

const detector = await poseDetection.createDetector(
  poseDetection.SupportedModels.MoveNet,
  detectorConfig,
  { fetchFunc: fetch }
)

Who will benefit with this feature? anyone porting tensoflowjs to deno trying to use the pose detection

Any Other info. deno --version

deno 1.24.1 (release, aarch64-apple-darwin)
v8 10.4.132.20
typescript 4.7.4

[vicary]

@andykais Thanks for the hint. It turns out to be me prematurely disposing a tensor before the promise is resolved, we could definitely use a better error message but that’s for another issue.

I can gladly say that my Deno fork basically works for all the use cases I have.

[haoyunfeix]

@andykais shader problems fixed with #7193, please look forward to the next version(should be https://cdn.skypack.dev/@tensorflow/tfjs-backend-webgpu@0.0.1-alpha.17, not published yet) can be runnable on your demo.