Prediction of numerical features

My yaml file looks like

training:
    epochs: 10
    learning_rate: 0.001
    batch_size: 128
    early_stop: 10

input_features:
    -
        name: lyrics
        type: text
        encoder: parallel_cnn
        level: word

output_features:
    -
        name: f1
        type: numerical
    -
        name: f2
        type: numerical

so I have two float features f1 and f2 (and so MSE will be used by default as loss) to be predicted over a input text, for which I’m using a parallel_cnn at word level.

After few epochs I’m getting a 0 accuracy.

╒════════════╤═══════════╤════════════╕
│ combined   │      loss │   accuracy │
╞════════════╪═══════════╪════════════╡
│ train      │ 1121.6427 │     0.0000 │
├────────────┼───────────┼────────────┤
│ vali       │ 1140.5157 │     0.0000 │
├────────────┼───────────┼────────────┤
│ test       │ 1136.8768 │     0.0000 │
╘════════════╧═══════════╧════════════╛

I get the same result when using for the input a different encoder like

input_features:
    -
        name: lyrics
        type: text
        encoder: rnn
        cell: lstm
        bidirectional: true

is the yaml output_features definition correct for these float values?

[w4nderlust]

Clipping and normalization were added to numerical features, so i consider this to be solved.

[w4nderlust]

@loretoparisi let me tray to understand your usecase batter. So those numbers that you want to output are between [0,1] but they are not probabilities of a binary classifier, is that correct?

Depending on that, one solution could be to add a preprocessing parameter like normalize_01 that performs this normalization at the data level (so it would work for both numerical inputs and numerical features. There could also be a normalize_zscore and a normalize_minmax normalization strategy, so probably it would be better to have a normalize parameter that by default is None but then you can pass a string with the name of the normalization strategy (01, maxmin, zscore) and it will adopt that strategy reading it from a normalization strategy registry.

This will work at the data level, but there wouldn’t be anything in the model to constraint it to produce a value in [0,1]. For that purpose one can think about writing a decoder that clips values before outputting them, or some other strategy (for instance applying a sigmoid). Adding a decoder should be pretty easy, the only difficulty is that sequence features for instance already have a machinery with a registry of decoders that are selected by their name, while numerical features don’t have that because so far there has only been one decoder. Adding it would be simple, and probably I should do it for all the features anyway.

What do you think?