Cannot include package that depends on tfjs-node in electron app

TensorFlow.js version

v1.5.2

Browser version

electron: ^6.0.0 configured with vue-cli and vue-cli-plugin-electron-builder vue: ^2.6.6 vue-cli-plugin-electron-builder: ^1.4.5

Describe the problem or feature request

I’ve seen similar issues to mine, but all the proposed solutions involve using webpack externals to ignore node modules. Which I don’t believe works in my case since my application needs to be packaged and run in electron.

Whenever I include a package that depends on @tensorflow/tfjs-node I get the following error:

This dependency was not found:

* aws-sdk in ./node_modules/node-pre-gyp/lib/info.js, ./node_modules/node-pre-gyp/lib/publish.js and 1 other

To install it, you can run: npm install --save aws-sdk

I first encountered this error when we took the tfjs-node code out of our electron app and created a separate repo / package for this piece of code, which we call the engine.

Before tfjs-node was required in package.json for our electron application. Everything worked correctly

After We repackaged the tf code into it’s own npm package called @samply-app/engine. So tfjs-node is now a dependency of @samply-app/engine, which is listed in the electron app’s package.json, but tfjs-node is no longer a direct dependency.

I have also tested this with other random packages that depend on tfjs-node, I get the same error each time.

Any insight would be great! I’ve tried the solutions i’ve seen posted about webpack externals and that just seems to lead to more modules going missing and my build fails.

Thanks!

Code to reproduce the bug / link to feature request

package.json - electron app

{
  "name": "samply-desktop",
  "version": "0.1.4",
  "private": true,
  "scripts": {
    "serve": "vue-cli-service electron:serve",
    "build": "vue-cli-service electron:build",
    "test": "yarn test:unit",
    "test:unit": "vue-cli-service test:unit",
    "lint": "vue-cli-service lint --fix",
    "publish": "vue-cli-service electron:build --publish always",
    "generate-icons": "electron-icon-builder --input=./public/icon.png --output=build --flatten",
    "generate-dev-icons": "electron-icon-builder --input=./public/dev-icon.png --output=build-dev --flatten"
  },
  "main": "background.js",
  "dependencies": {
    "@mdi/js": "^3.8.95",
    "@samply-app/engine": "^0.1.27",
    "@types/d3": "^5.7.2",
    "@types/howler": "^2.1.2",
    "core-js": "^3.4.4",
    "d3": "^5.9.2",
    "direct-vuex": "^0.9.11",
    "electron-updater": "^4.0.6",
    "eslint-plugin-html": "^5.0.3",
    "firebase": "^7.7.0",
    "glob": "^7.1.3",
    "howler": "^2.1.3",
    "material-design-icons-iconfont": "^4.0.5",
    "three": "^0.107.0",
    "vue": "^2.6.6",
    "vue-router": "^3.0.1",
    "vuetify": "^2.1.5",
    "vuex": "^3.0.1",
    "vuex-persistedstate": "^2.5.4"
  },
  "devDependencies": {
    "@types/jest": "^24.0.19",
    "@vue/cli-plugin-babel": "^4.1.2",
    "@vue/cli-plugin-eslint": "^4.1.2",
    "@vue/cli-plugin-typescript": "^4.1.2",
    "@vue/cli-plugin-unit-jest": "^4.1.2",
    "@vue/cli-service": "^4.1.2",
    "@vue/eslint-config-airbnb": "^4.0.0",
    "@vue/eslint-config-typescript": "^4.0.0",
    "@vue/test-utils": "1.0.0-beta.29",
    "babel-core": "7.0.0-bridge.0",
    "babel-eslint": "^10.0.1",
    "babel-jest": "^23.6.0",
    "electron": "^6.0.0",
    "electron-icon-builder": "^1.0.0",
    "eslint": "^5.8.0",
    "eslint-plugin-vue": "^5.0.0",
    "eslint-plugin-vuetify": "^1.0.0-beta.3",
    "sass": "^1.22.9",
    "sass-loader": "^8.0.0",
    "style-loader": "^0.23.1",
    "typescript": "~3.5.3",
    "vue-cli-plugin-electron-builder": "^1.4.5",
    "vue-template-compiler": "^2.5.21"
  }
}

package.json - “@samply-app/engine”

{
  "name": "@samply-app/engine",
  "version": "0.1.27",
  "description": "Analyzes audio waveforms",
  "main": "dist/index.js",
  "types": "dist/index.d.ts",
  "repository": "https://github.com/samply-app/engine.git",
  "author": "eschirtz <[e-mail]>",
  "license": "MIT",
  "private": false,
  "files": [
    "dist",
    "model"
  ],
  "scripts": {
    "build": "tsc",
    "dev": "tsc && node ./sandbox"
  },
  "dependencies": {
    "@tensorflow/tfjs-node": "^1.5.2",
    "audio-decode": "^1.4.0"
  }
}

[rthadur]

@kangyizhang gentle ping if you have any thoughts on this ?

[eschirtz]

Alright, so I’ve done some more debugging. Found a fix for our project, but not a perfect one. It still requires that I include @tensorflow/tfjs-node as a direct dependency of our project, not just the package that actually uses tfjs.

Turns out our Electron (6.1.9) uses Node v12.4.0, which is too old to use N-API 5. Looks like N-API 5 requires Node v12.11.0. It appears that the latest version of tfjs-node v1.7.0 uses N-API 5, where as our older v1.0.2 uses N-API 4.

If this is true, I’m a little confused why tfjs-node did not get a major version bump, since it is a breaking change? Should this be backwards compatible?

I then ended up downgrading our engine package to use tfjs-node v1.0.2 in order to get around the N-API 5 errors. However, when I removed tfjs-node as a direct dependency from our app (so that it was only a dependency of engine), I got the follow warnings.

Registration of backend tensorflow failed

The "path" argument must be of type string

And the backend of tf was web-gl and our analysis took about twice as long.

Then when I yarn add @tensorflow/tfjs-node@1.0.2 in our application (direct dependency), the error went away, my backend switched back to tensorflow and our analysis ran at full speed.

I hope some of this can shed some light on the issue, please let me know your thoughts!

Thanks!