Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

bug(logger): wandb fails on sweep

See original GitHub issue

🐛 Bug

When using wandb sweeps for hyperparameters search, I get this error:

wandb: ERROR Attempted to change value of key “dropout_std” from 0.030424838979365657 to 0.030424838979365654

The reason is I ran:

wandb_logger.log_hyperparams(params)

Which I guess has some problem with floating-point numbers in high accuracy?

Issue Analytics

State:
Created 3 years ago
Comments:6 (6 by maintainers)

Top GitHub Comments

1reaction

borisdaymacommented, Mar 30, 2020

The problem is that it tries to log this value twice and is probably called before automatically by pytorch-lightning. The callback will automatically log every parameter which is in pl.LightningModule.params (where you probably already have the dropout).

See an example of using pytorch-lightning with wandb (including sweeps) here: https://github.com/borisdayma/lightning-kitti

I’ll be adding it to the pytorch-lightning repo later but still need to push a PR related to the watch method for it to work properly.

0reactions

borisdaymacommented, Apr 16, 2020

I added a fix. @AmitMY @amoudgl Feel free to test it with your sweeps and let me know if there’s still an error.

Top Results From Across the Web

bug(logger): wandb fails on sweep · Issue #1290 - GitHub

Bug When using wandb sweeps for hyperparameters search, I get this error: wandb: ERROR Attempted to change value of key "dropout_std" from ...

Agent bug? File not found error - W&B Help - WandB community

It looks like your sweep page is getting created from the image that you had sent and I have a few questions. Is...

wandb Changelog - PyUp.io

Fixes Sweeps on Launch CLI launch config, relpath by hu-po in https://github.com/wandb/wandb/pull/4073 * Fixes broken Launch apikey error message by ...

Weights & Biases sweep cannot import modules with pytorch ...

#!/usr/bin/env python import wandb from utils import get_config ... import Trainer from pytorch_lightning.loggers import WandbLogger from ...

attributeerror: module 'wandb' has no attribute 'run' - You.com

I have searched the YOLOv5 issues and found no similar bug report. ... When I run a sweep using wandb agent [sweepID] I...