Record history of hparams metrics

I’m unsure how to record metrics during training such that when I run add_hparams() at the end, the show metrics graphs in the HPARAMS tab contains more than a single value. What I would like to do is something like this:

w.add_scalar("loss", 10, 0)
w.add_scalar("loss", 11, 1)
w.add_scalar("loss", 12, 2)
w.add_hparams({'lr': X, 'bsize': Y, 'n_hidden': Z}, {'loss': 13})

And then the show metrics graph would have the four points [10,11,12,13] plotted for the loss metric.

Thanks!

[asford]

@rohaldbUni Then TensorBoard HParams plugin will aggregate and report scalars from your run into the reporting page, you just need to manually write the experimental overview into the tensorboard event stream. For your example:

experiment, start_summary, end_summary = tensorboardX.summary.hparams(
    {'lr': X, 'bsize': Y, 'n_hidden': Z}, {'loss': None}
)

w.file_writer.add_summary(experiment)
w.file_writer.add_summary(start_summary)

w.add_scalar("loss", 10, 0)
w.add_scalar("loss", 11, 1)
w.add_scalar("loss", 12, 2)
w.add_scalar("loss", 13, 2)

Should report the full “loss” trace on the hparams summary page.

[asford]

@lanpa Is this repo the primary development point for tensorboardX, or is the module being folded into pytorch mainline development? I’ve noticed a bit of discussion there on https://github.com/pytorch/pytorch/pull/23134 and https://github.com/pytorch/pytorch/issues/16838.

I’m currently using a method like the one above for hparam reporting in pytorch, but I’d be happy to expand and document the current tensorboardX interface to cover this use case.