F1 score support for objective

Today objective = “val_f1” returns an error Failed to train : <class ‘ValueError’> : Could not infer optimization direction (“min” or “max”) for unknown metric “val_f1”. Please specify the objective asa kerastuner.Objective, for example kerastuner.Objective("val_f1", direction="min").

[natanrajch]

Had this same issue while running latest version of autokeras in Colab environment. While using this f1 custom objective, the object’s .fit() worked OK, but failed to .predict() or .export_model() after training. Keras was demanding the custom objects, and they weren’t being passed on.

Fixed it by changing the StructuredDataClassifierTuner function like this, I leave it here for reference of whomever might find this useful:

my_custom_objects={'f1_score_cust': f1_score_custom}
my_custom_objects.update(ak.CUSTOM_OBJECTS)

from keras_tuner.engine import hypermodel as hm_module
def get_best_model_custom(self):
  with hm_module.maybe_distribute(self.distribution_strategy):
    model = tf.keras.models.load_model(self.best_model_path, custom_objects=my_custom_objects)
  return model
autok.tuner.get_best_model = get_best_model_custom.__get__(autok.tuner,ak.tuners.task_specific.StructuredDataClassifierTuner)

Here f1_score_custom is a custom function, ak is autokeras, and autok is an StructuredDataClassifier instance.

[haifeng-jin]

I am working on it. It seems it is a bug that it always use val_loss as objective.