wandb loses "global_step" in PyTorch event_writer
See original GitHub issueThere doesn’t seem to be a correspondence between global_step
and step
in my graphs. I’m using wandb through eventwriter API in PyTorch. Debugging a bit, it seems
step
gets converted to global_step
here
https://github.com/wandb/client/blob/6417dd926abe76dbb7c56e7017d2ee7d1c918eb5/wandb/tensorboard/__init__.py#L205
Then wandb.log
gets called with this dict, it doesn’t see step
so it assigns one automatically
Custom global step is useful to compare data efficiency consistency across runs – using “forward calls” as x-axis means your curves look 2x better when doubling batch size or number of workers. Using global data counter for steps gives easier to interpret curves @vanpelt
Issue Analytics
- State:
- Created 4 years ago
- Comments:6 (2 by maintainers)
Top Results From Across the Web
wandb loses "global_step" in PyTorch event_writer · Issue #619
I'm using wandb through eventwriter API in PyTorch. ... Custom global step is useful to compare data efficiency consistency across runs ...
Read more >PyTorch - Documentation - Weights & Biases - Wandb
W&B provides first class support for PyTorch, from logging gradients to profiling your code on the CPU and GPU.
Read more >wandb_logger — PyTorch-Ignite v0.4.10 Documentation
See documentation # on wandb.init for details. wandb_logger = WandBLogger( ... global step transform function to output a desired global step.
Read more >Supercharge your Training with PyTorch Lightning + Weights ...
In this video, Weights & Biases Deep Learning Educator Charles Frye demonstrates how to use PyTorch Lightning with W&B to build ML pipelines ......
Read more >Experiment Logging with TensorBoard and wandb
To start with PyTorch version of TensorBoard, just install it from ... in a subgroup tag="loss" ) # incrementing the global step (number...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Yeah, we look for any metrics that are monotonically increasing and make them available as x-axis options. MLFlow, Tensorboard, and others allow you to go back in time during training and we don’t. By putting the step in metrics we’re punting the missing feature by allowing users to log historic steps at the cost of using them as an x-axis (but still being able to export via the python ap). The big rewrite will enable this behaviour. We’re seeing a bunch of users syncing tensorboard, so we should atleast default to global_step as the x-axis in this case.
Thanks for the info, closing it as duplicate of https://github.com/wandb/client/issues/613