Multiple runs in notebook cells are not properly synced (wrong "state" for all but last)

wandb --version && python --version && uname

• Weights and Biases version: 0.8.36
• Python version: Python 3.6.9 :: Anaconda, Inc.
• Operating System: Linux (Ubuntu 18.04) and macOS (10.15.4)

Description

I’m trying to execute several runs from a single cell in a Jupyter Notebook (for example, doing cross-validation with each split as a separate run.)

All runs execute and their results are logged, however, only the last run shows up as being finished — all earlier runs still show as running (until they are registered as crashed some time later).

I would expect all successfully completed runs to have state finished.

(When running from a script, everything works fine.)

What I Did

A small example to illustrate the problem:

import wandb
import time

for i in range(10):
    with wandb.init(project="sandbox", name=f"wandb-test-{i + 1}", reinit=True) as run:
        run.summary['accuracy'] = (i + 1) / 10
        time.sleep(5)

Running this in a notebook cell results in the last run (“wandb-test-10”) having state finished, but runs “wandb-test-1” through “wandb-test-9” still show up as running.

If instead the same code is run outside of a notebook in a separate script, all runs are correctly registered as being finished.

[tyomhak]

This is a bug, that we’re working on. Running the same code in a script through a cmd should work just fine, but there seems to be an issue with running experiments in a loop in a notebook. Sorry for the inconvenience.

Also, I would suggest upgrading your wandb. The latest version is 0.9.3

[ariG23498]

Hey folks in the past year we’ve majorly reworked the CLI and UI for Weights & Biases. We’re closing some stale issues. Please comment to reopen.