Unexpected Behavior unable to save training data (offline example)

I’m new to this so I hope this is a small issue, nevertheless will be thorough.

Goal: I want to create a sample of training recommendations offline, load it into a web app and run it without any training on the browser. However, I want to be able to submit data back to a server where the model can train on new additional data. Inspired by BrainJSReactNative.

What is wrong?

Seeing some unexpected behavior when trying to train data offline with newly appended training data. The workaround is retraining the model from scratch with new data, which seems odd.

I’m using this example from the React Native repo to do the following:

1. Import existing trained data (Trained model with lyrics by ACDC and Lee Kernaghan)
2. Train Data with new training data (same as old training data + Slayer lyrics)
3. Run Model with Slayer lyrics which should score slayer the highest

Problem(s):

1. “Slayer” is not part of the output at all
2. “Slayer” is not added to the model weights, in fact, the training model stays virtually the same as before.

Training Data:

• 4 ACDC lyrics (pre-trained)
• 10 Lee Kernaghan lyrics (pre-trained)
• 2 Slayer lyrics (new)

Sample input:

// Example input - pre-encoded - each word is encoded to a number when it hits the train runtime
{ input: `Up in the morning and out to school
        The teacher is teaching the golden rule
        American history and practical maths`,
    output: {ACDC: 1}}

Training:

net.train(processTrainingData(trainingData), {
  iterations: 20000,
  learningRate: 0.1,
  logPeriod: 1,
  momentum: 0.1,
  log: true,
  errorThresh: 0.00000000000001,
  timeout: Infinity,
  callback: ((state) => {
    console.log(state);
  })
});

Running:

let results = net.run(encode(`Surgery, with no anesthesia
    Feel the knife pierce you intensely
    Inferior, no use to mankind
    Strapped down screaming out to die
    Angel of Death
    Monarch to the kingdom of the dead
    Infamous butcher,
    Angel of Death`));

console.log(results);

Output:

Saving:

fs.writeFileSync('trained-model.json', JSON.stringify(net.toJSON()));

Slayer is not added to weights, nothing has changed.

Where does it happen?

Running this code when adding training data.

How do we replicate the issue?

Running this code when adding training data. Example training data appended to trainingData:

  {
    input: `Surgery, with no anesthesia
    Feel the knife pierce you intensely
    Inferior, no use to mankind
    Strapped down screaming out to die
    Angel of Death
    Monarch to the kingdom of the dead
    Infamous butcher,
    Angel of Death`,
    output: {Slayer: 1}
  },
  {
    input: `Crowded tombs
    Sickening ways to achieve
    The [Explicit]
    Seas of blood, bury life
    Smell your death as it burns
    Deep inside of you
    Abacinate, eyes that bleed
    Praying for the end of
    Your wide awake nightmare
    Wings of pain, reach out for you
    His face of death staring down,
    Your blood running cold
    Injecting cells, dying eyes
    Feeding on the screams of
    The mutants he's creating
    Pathetic harmless victims
    Left to die
    Rancid Angel of Death
    Flying free`,
    output: {Slayer: 1}
  }

How important is this (1-5)?

5 - can’t do offline training, have to train models from scratch if training data changes. Which is a pain.

Expected behavior (i.e. solution)

The model should append new training data. Shouldn’t have to start from scratch every time.

Other Comments

Searched through the issues, was unable to find a solution

[robertleeplummerjr]

Booya! https://jsfiddle.net/robertleeplummerjr/dhn7k1yq/10/

First time I ran it, I got: “obby cerrin”, with zero configuration, and probably overfitting. In short, they (we) just need to use a more applicable net is all.

[robertleeplummerjr]

There are still some missing layers with the GPU LSTM, not yet, but SOON.