Image recognition

Thanks a lot guys for creating such awesome tool for neural networks for javascript community. I have tried brain.js for creating NN for image recognition. Super-simple one - do the image contains samoyed or no.

And now I am a bit struggled. Cannot find anything about image recognition in the repo (only #176 but nothing was helpful from here)

• Does the brain.js good solution for image recognition?
• How to proceed?
• How to prepare image data?
• Why it takes so much time?

I am looking for answers and would be good if you will create some example 😃 I am now doing something like this but it doesn’t work anyway. 64x64 images processing deadly slow. For different image resolutions I am getting error: NaN.

[robertleeplummerjr]

Guys, I’ve added a table of percentages to show where we are with v2: https://github.com/BrainJS/brain.js/wiki/Roadmap#v2-status

This may help at least clarify where things are. I found a bug two days ago with GPU.js that I’m addressing that has caused a few more days delay for v2 of GPU.js release. I’ll have it out soon though.

[robertleeplummerjr]

Does the brain.js good solution for image recognition?

TL;DR Currently, in v1, it is not ideal. It uses CPU.

TS;DR This is the single most active part of what is going on in Brian.js. In fact the not yet working example for mnist ( https://github.com/BrainJS/mnist-demo/blob/gh-pages/index.js#L1 ) is sitting, waiting, and ready. I’ve been focused on bringing Node GPU into the mix, via the dependency GPU.js here: https://github.com/gpujs/gpu.js/tree/gl-headless-experimental Which adds to compliment brain.js:

• Typescript support
• Node GPU support (easy for OSX, and Linux, yet to have working easily with Windows)
• Promises
• Much cleaner API
• More thoroughly tested

Already supported are:

• Direct image inputs
• Direct arrays of images inputs
• Tons of other inputs, and soon to have direct VIDEO input

YES, direct Video into a neural network!

The main features that are needed are convolution layers, which consist of convolution, pool, fullyConnected, & softMax. After they are unit tested fully, there will be a bit of API work to be done, and the net will be in a spot to release for v2. So ultimately things are very close, but I needed to take some time to push the GPU.js side of things to get it to the point it could be an enterprise platform for GPU programming with simple javascript that is built inside brain.js.

• v2 Branch https://github.com/BrainJS/brain.js/tree/develop
• Unit tests state: https://travis-ci.com/BrainJS/brain.js-cnn-integrity

How to proceed?

You have a choice:

• The easy and most traveled road:You could use the tools that are better suited for the project you are working on, until brain is ready and then you could use brain.js.
• “The Road Not Taken”, would be to become active in brain.js, and help push v2 out the door, so we can finally use it. This is the road which I’ve taken, and the one that “has made all the difference”.

How to prepare image data?

Currently you’d need to convert the values to arrays.

In v2, you’d just feed the image directly into the neural network.

Why it takes so much time?

If you are talking about the existing implementation in brain.js, it is because the CPU is, in short… Very slow (one at a time) when you compare its ability to process a lot of data. There are ways to get it faster, but even on modern hardware, even if we ported the whole things to something that could run on multi CPU’s, it’d be around 4ish times faster on a modern i7. If we converted it to WebAssembly, we could push that by maybe 1.5 times. But as soon as we go to the GPU, on a basic no frills GPU, we can easily get 5 to 10 times faster without using pipelines (textures that aren’t transfered back to the CPU, they stay on GPU). With textures, we can get 12 to 50 times faster,and with a decent video card, that number rises FAST, (NOTE: I’ve seen it go as high as 300 times faster on an IMAC, though this isn’t entirely accurate because in texture mode it processes somewhat async-ish, so we’re still working out how well to test it for speed, but it is CRAZY fast, is the point).

If you are referring to the speed at which I’ve been building these components, however (which I don’t think you are, but I’d like to take a moment to clarify) I didn’t think it’d take this long either.

I’ve mentioned (possibly ranted) about this , before, but right now, literally today or tomorrow GPU.js v2 will be released if I can find an hour or so.

It has been one of the hardest things I’ve ever worked on professionally to take the various complexities of all the required components and put them together, here is a small part:

• Why current Javascript Neural Networks have mostly been abandoned? CPU isn’t fast enough, GPU is needed.
• What is needed to solve their next iteration? Keep the language in javascript, but utilize GPU EVERYWHERE, not just on browsers.
• To just say, “Ok, that was a problem, but after I get done with it, there will no longer be one” and join a hopelessly long stack of tasks.
• To find the project that had the idea of targeting GPU from javascript, without having to write GPU code. GPU.js.
  • Joining their project (gpu.js), seeing it was somewhat abandoned.
  • Understanding how to write GPU equivalent code for javascript.
  • Create means of debugging javascript code.
  • Making the CPU implementation faster, so CPU code doesn’t take a performance penalty.
  • Rewriting the project essentially from scratch, hoping I can open a PR and it isn’t laughed at, and possible I join the team.
  • Joining the team
  • Learning how to parse glsl from javascript
  • 100 other things, while working on brain.js and gpu.js
  • skip forward to working in the c++ guts of Google Angle, and why it isn’t working with certain outputs correctly
  • compiling headless-gl and Google Angle (a fantastic project, but no binaries? Come on Google!)
• Propose a brain.js v2 api
• Work relentlessly on brain.js v2 api, building and discovering what is needed to make it super simple.

Many other things happened, and many people have assisted along the way. The invitation is there for you to join as well. It could make all the difference.