Unknown metric error when loading a model trained with stateful metrics

Hi everyone,

I’m trying to load a saved model which has been trained with a custom stateful metric (object of the class ValidAccuracy using the code

dependencies = {
     'ValidAccuracy': ValidAccuracy
}
model = keras.models.load_model(model_file, custom_objects=dependencies)

and I get the error ValueError: Unknown metric function: {'config': {'batch_size': 32, 'name': 'valid_accuracy', 'trainable': False}, 'class_name': <class 'spearec.run.metrics.ValidAccuracy'>}.

Is there any solution or workaround?

[brge17]

Stateful Metrics are custom layers:

If you really want to load a custom model with custom layers: https://github.com/keras-team/keras/issues/4871.

Or you can just do load_weights.

I’m of the opinion that load_model shouldn’t even exist. Have a function that defines your network, train your model, save the weights, create the network, load your weights, predict.

You shouldn’t need the training metrics, or the optimizer to do predictions.

[brge17]

dependencies = {
    'valid_accuracy': ValidAccuracy
}

should solve the issue.

Stateful metric inherits from layer and forces it to lower_case_underscore. As per the error message, it is looking for valid_accuracy and not ValidAccuracy.