Hparams with Estimators

System information

• TensorFlow and Tensorboard version (you are using): 2.1.0
• Are you willing to contribute it (Yes/No): Yes

Describe the feature and the current behavior/state. Hparams cannot be used with estimators in tensorflow 2. There’s a KerasCallback class that logs the hparams when using Keras model.fit, however this is not usable when using Estimator API.

There could be a feature in the form of a callback or hook that can be attached to estimators (with TrainSpec or EvalSpec) which will log hparams metrics against the hparams for the current trial.

Who will benefit with this feature? Those who rely on Estimator API and tensorboard will benefit.

[saravanabalagi]

The culprit seems to be

# This doesn't seem to compatible with Estimator API
hp.hparams_config(hparams=hparams_list, metrics=metrics_to_monitor)

At the moment, I guess it’s not possible to do use hparam_config(...) with Estimator API. Here’s a workaround until it becomes compatible.

Simply call hparams logs all metrics logged with tf.summary. Then in tensorflow, you can filter only the metrics you need and then compare trials.

with tf.summary.create_file_writer(train_folder).as_default():
    # params is a dict which contains
    # { 'learning_rate': 0.001, 'distance_margin': 0.5,...}
    hp.hparams(hparams=params))

[saravanabalagi]

@jarednielsen yes, it automatically logged the metrics whichever I logged using summary. So if you didn’t have summary.scalar, you may not see anything.