Metric present but marked as <null> (Parallel Coordinates)

wandb --version && python --version && uname

• Weights and Biases version: 0.9.4
• Python version: 3.6.9
• Operating System: Linux (Ubuntu 18.04.2)

Description

Eventhough I have values for my accuracy metric (the graph is not empty), the Parallel Coordinates graph doesn’t show the final value (all runs are marked as <null>).

What I Did

I guess I know where the problem might be because my logs are a little bit strange. I am building my model in 2 times :

• First, I train an autoencoder, logging loss and val_loss metrics to Wandb plus an additional accuracy which value is 0 (needed by keras-tuner library to perform the hyperparameter search) at the end of each epoch.
• Then, I create an anomaly detection model from the autoencoder (determine a threshold and everything) and evaluate its performances on a supervised task which returns an accuracy value (and few others metrics, as FalsePositive, …) that I log to Wandb.

This double logging thing make my last accuracy value logged one step after many other metrics (ie, if loss and val_loss were logged during the first 156 steps, the last accuracy value will be on the 157th step).

For information, during the autoencoder’s training the accuracy value is correctly shown as 0 and get wiped out afterwards.

[pierreelliott]

Link sent!

[jarridrb]

Want to note that I’m getting a similar issue as well. I can look into it more in the next few days to post more details, but the short version of things is that I have a number of runs which have their summary metrics filled while training progresses, but once the trial has finished the summary metrics disappear similar to the images above. I’m using the wandb integration with Ray Tune which potentially could be a culprit for the issue, but I’m not certain. One thing to note is that I checked wandb-summary.json for a couple of the affected runs and the metrics which I’m looking to track are present there even after the run finishes, but not on the wandb web UI.