Allow alternative GLContexts to be set globally.

Currently, canvas_util.ts has some caching of WebGL context versions with a retrieval function that attempts to load a WebGL context by creating a new canvas DOM element.

Ideally, we have a flag or global override to set the WebGL/GL context object.

For instance, setting WEBGL_FENCE_API_ENABLED to false will result in this exception when running outside of the browser:

(node:89462) UnhandledPromiseRejectionWarning: ReferenceError: document is not defined
    at getWebGLRenderingContext (/Users/kreeger/workspace/tfjs-backend-nodegl/node_modules/@tensorflow/tfjs-core/src/backends/webgl/canvas_util.ts:63:18)
    at getWebGLContext (/Users/kreeger/workspace/tfjs-backend-nodegl/node_modules/@tensorflow/tfjs-core/src/backends/webgl/canvas_util.ts:37:30)
    at Object.getWebGLContext (/Users/kreeger/workspace/tfjs-backend-nodegl/node_modules/@tensorflow/tfjs-core/src/backends/webgl/canvas_util.ts:42:12)
    at Object.getWebGLDisjointQueryTimerVersion (/Users/kreeger/workspace/tfjs-backend-nodegl/node_modules/@tensorflow/tfjs-core/src/backends/webgl/webgl_util.ts:516:14)
    at Object.evaluationFn (/Users/kreeger/workspace/tfjs-backend-nodegl/node_modules/@tensorflow/tfjs-core/src/backends/webgl/flags_webgl.ts:104:21)
    at Environment.evaluateFlag (/Users/kreeger/workspace/tfjs-backend-nodegl/node_modules/@tensorflow/tfjs-core/src/environment.ts:98:40)
    at Environment.get (/Users/kreeger/workspace/tfjs-backend-nodegl/node_modules/@tensorflow/tfjs-core/src/environment.ts:61:33)
    at Environment.getNumber (/Users/kreeger/workspace/tfjs-backend-nodegl/node_modules/@tensorflow/tfjs-core/src/environment.ts:67:17)
    at GPGPUContext.createFence (/Users/kreeger/workspace/tfjs-backend-nodegl/node_modules/@tensorflow/tfjs-core/src/backends/webgl/gpgpu_context.ts:254:13)
    at GPGPUContext.createAndWaitForFence (/Users/kreeger/workspace/tfjs-backend-nodegl/node_modules/@tensorflow/tfjs-core/src/backends/webgl/gpgpu_context.ts:232:31)

I tried importing setWebGLContext but the key as a number does not work on contexts. I’ll whip up a patch tomorrow for this.

[yuccai]

Hi, I wonder what the current state of tfjs is regarding setting a custom webgl context.

I’m using :

    "@tensorflow/tfjs": "~2.8.4",
    "@tensorflow/tfjs-backend-webgl": "~2.8.4"

I want to read output webgl texture of tensorflow js webgl backend in a 3d rendering context like this way :

  const my3dTexture = new Texture(...);
  const output = model.predict();
  const { texture: tfjsWebGLTexture } = tf.backend().texData.get(output.dataId);
  my3dTexture._texture._webGLTexture = tfjsWebGLTexture;
  console.log(my3dTexture.readPixels());

The my3dTexture.readPixels() function acts like output.dataSync() but it tries to read the data from my 3d rendering context. This piece of code throws the following error : WebGL: INVALID_OPERATION: framebufferTexture2D: object does not belong to this context. This is the reason why I want to use the same context.

I tried different solutions based on this thread but I encounter different errors :

• First try, change the gl context

import * as tf from "@tensorflow/tfjs";
import { setWebGLContext } from "@tensorflow/tfjs-backend-webgl";

const gl = canvas.getContext("webgl2");
setWebGLContext(2, gl);
await tf.setBackend("webgl");
gl.enable(gl.SCISSOR_TEST);
gl.scissor(0, 0, canvas.width, canvas.height);

In this case, the gl context of tfjs is not equal to my 3d context : gl === tf.backend().gpgpu.gl is false so the main error is still thrown. I tried adding tf.backend().gpgpu.gl = gl; but in this case, errors are thrown from tfjs side (object does not belong to this context)

• Second try, create new backend

import * as tf from "@tensorflow/tfjs";
import { GPGPUContext, MathBackendWebGL } from "@tensorflow/tfjs-backend-webgl";

  const gl = canvas.getContext("webgl2");
  const gpgpu = new GPGPUContext(gl);
  tf.registerBackend("custom-webgl", () => new MathBackendWebGL(gpgpu));
  await tf.setBackend("custom-webgl");

In this case, tfjs context and my 3d context are the same : gl === tf.backend().gpgpu.gl is true But a basic tensor operation like tf.square(4) throws the following error : Error running Square: Neither modular kernel nor forward func passed.

As a recap, what I need is to be able to directly use a tfjs webgl texture (generated in the tfjs context) in an other WebGL context. I tried two different ways to set the same context but it fails. Any thoughts ?

[pmer]

My apologies, butler, but I think there is still an interest in waiting to hear more on this issue.