Running multiple intances in separate workers

To get help from the community, we encourage using Stack Overflow and the tensorflow.js tag.

TensorFlow.js version

tfjs-node 1.2.9

Browser version

Node.js 10.16.0

Describe the problem or feature request

Trying to create a new model in each thread for inference, but only the first one works. After the first one the following threads fail with this error:

I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that thError: Error: Module did not self-register.
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:805:18)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)
    at Module.require (internal/modules/cjs/loader.js:690:17)
    at require (internal/modules/cjs/helpers.js:25:18)
    at Object.<anonymous> (\node_modules\@tensorflow\tfjs-node\dist\index.js:44:16)
    at Module._compile (internal/modules/cjs/loader.js:776:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:787:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
is TensorFlow binary was not compiled to use: AVX2

Is it possible to run multiple models/sessions at the same time on the same CPU/GPU?

I am using another (minified) library that loads an existing tf model and creates the instance, but the error is whithin the TensorFlow code, not of that library, so I can not provide any relevant code.

[bkulcsar]

I have the same issue. Stack: Linux & Node 14.15.1 Repro steps are:

1. Create worker by worker_threads in Node.js
2. Use @tensorflow/tfjs-node in the worker.
3. Run the worker - first worker is running properly
4. Run another instance of the worker, it will fail with the exception: Error: Module did not self-register: @tensorflow/tfjs-node

[josephrocca]

I have a related problem - I’m trying to create multiple instances of a model using the wasm backend - each in their own worker. It seems like, currently, all instances are put into the same worker, so only one model can execute at a time, even though there are several of them initialized.

It would be great if it were possible to specify that tfjs should create a new worker per model.

(Also, kind of strange that even if I set tf.wasm.setThreadsCount(1) before the await tf.setBackend("wasm") call, tfjs still creates 8 workers, even when only initialising a single model. Please let me know if this is unexpected/incorrect behavior and I will file a separate issue with a minimal reproduction.)