Training with Raw Image data as inputs

What is wrong?

Training error does not go below 1, and I think this is because my inputs strings have different lengths.

(currently i am using base64 encoded image data as input to the network, is this OK?)

Where does it happen?

In the a new brain.recurrent.RNN() when trying to run a net in node.js on my Windows 10

How do we replicate the issue?

Git Source here: https://github.com/mubaidr/image-parsing/blob/master/src/index.js#L45

How important is this (1-5)?

3

Expected behavior (i.e. solution)

Any suggestions to make input data to be same length for all images or maybe i am doing something wrong?

Other Comments

Thanks for the awesome work!

[jusso-dev]

I don’t have an answer to this but + 1 for the gif

[robertleeplummerjr]

Wouldn’t it be possbible to use r g b values from 0 to 1 in a neural network?

In gpu.js (which is the engine for brain.js v2) we’ve built it so you can feed images in directly: https://github.com/gpujs/gpu.js#supported-input-types

const myFunc = gpu.createKernel(function(image) {
    const pixel = image[this.thread.y][this.thread.x];
    this.color(pixel[0], pixel[1], pixel[2], pixel[3]);
})
  .setGraphical(true)
  .setOutput([100]);

const image = document.createElement('img');
image.src = 'my/image/source.png';
image.onload = () => {
  myFunc(image);
  // Result: colorful image
};

If we brought this functionality into v2 of brain.js, it would likely be:

1. after the release of brain.js v2
2. after the release of gpu.js v2 with OpenCL support

I’ve personally hoped for it for some time, and I think, although challenging, it would be terribly useful. However, we’d need to plan for how that api would look and ensure it’d be useful for real world scenarios.

However, @mubaidr raises a good point:

In my case images were simpler and color information was not needed.

Most scenarios really don’t need color data. But I figure, if you are going to make something useful…

Why not make it easy?

(Caution: possibly overly simplified use case)

net.run(<img />);

// <img /> = document.createElement('img');