Ignore outliers setting can hide entire series

y axis is not automatically adjusted (and cannot be manually adjusted either) to fit multiple runs data series on one plot.

Example: [files to reproduce] There are 2 runs (train and validation) and both contain the same variable tower_0/costs/error.

Till step ~2.5k, when both runs are selected, you can only see train run data. It seems to me this is a bug, I would expect the y axis to be automatically adjusted to see both runs data series. The tooltip also doesn’t show validation run data when I inspect the graph. However the validation run data is there and shows up once I deselect the ‘train’ run. (Pay attention to the y axis values between plots.) Only later once the validation error drops below initial train error you can see both runs at the same time, but you still cannot zoom out to see both data series in the entire range of x axis. That is certainly a missing feature.

Information which may be relevant: -run train starts logging first in terms of steps and time -run validation starts running and logging at arbitrary step, i.e. step count doesn’t start at 0 I think this may be relevant to how tensorboard decides what axis limits to plot.

All in all it’s an annoying bug, because I thought that the validation summaries are just not saved since I could not see them, and it took a while to realize they actually are saved, just not plotted.

I’m using Tensorflow 1.2.0-rc0

[decentralion]

Glad you liked it 😃 Since we can’t reproduce this, and it looks like it may be related to old versions of TB, I’m going to close this for now. Please feel free to re-open if you find a repro and we’ll dig into it.

[talesa]

I have not upgraded TF on purpose at any point when that was happening and I reckon upgrading is not happening automatically/spontaneously either. And I am pretty sure I’ve tested it for longer than an hour, so I’m slightly doubtful it was a caching issue. Also, lol’ed at shouldiblamecaching.com