Chart domains should update as training progresses

TensorBoard version: https://pypi.org/project/tb-nightly/1.14.0a20190301/

Repro steps:

1. Train with Keras.fit() using the TensorBoard callback for 5 epochs
2. Open TensorBoard and look at the Scalars dashboard. See the loss and accuracy charts for epochs 1-5.
3. Continue training with Keras.fit() using the TensorBoard callback for another 5 epochs

Expected: As long as I didn’t manually change the zoom of the charts, the chart should rescale to show me the new data (of the new epochs).

Actual: The data from the continued training gets added to the chart but I can see that it goes off the chart.

Relevant code (assuming notebook environment):

model.fit(x=x_train, 
          y=y_train, 
          epochs=5, 
          validation_data=(x_test, y_test), 
          callbacks=[tensorboard_callback])

%tensorboard --logdir logs

model.fit(x=x_train, 
          y=y_train, 
          epochs=10,
          initial_epoch=5,
          validation_data=(x_test, y_test), 
          callbacks=[tensorboard_callback])

Image of current behavior:

Note: this is not specific to Keras.fit().

[makslevental]

yea the fact that domain doesn’t refit is definitely confusing to me.

[nfelt]

Thanks for your continued interest, folks. I’m happy to announce that our experimental new Time Series dashboard in TensorBoard 2.4.0 (most recent release) addresses the “refit on update” part of the issue. The new dashboard lets you view scalars, histograms, and images side-by-side in a combined view, and the scalar charts in the Time Series dashboard should auto-adjust the chart domain when new data is loaded as long as you haven’t changed the pan or zoom settings.

Please give that a try if you can and let us know if you have feedback. We recognize the “pan horizontally on update” part of the FR is not addressed by these changes, but as discussed above, it’s a little less clear how to automatically do the right thing in that case. For example, if the new data shifts in the Y axis such that none of it would be within the zoomed view, showing a blank chart could be confusing, but panning the chart to show the data could also be confusing.

We can consider at least showing a UI indication of new data to display in that case; it would also be helpful to know in more detail about exactly the behavior you would prefer to see there, and what is leading you to zoom in on the chart in the first place, since there might be other ways to approach this.