question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

System metrics: customize logging frequency

See original GitHub issue

wandb --version && python --version && uname

  • Weights and Biases version: version 0.9.1
  • Python version: Python 3.7.4
  • Operating System: MAC OS

Description

Hi there! From the documentation, I find that default metrics will be averaged every 30 seconds. I want to ask if we have any approach to change the time interval. I hope that users can have the option to choose by themselves. For example, if I have a program that runs for about 3 minutes, we might want to record system metrics every 5 or 10 seconds. If not, is there any way to add some code to make a control outside of wandb package. Thank you!

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:8 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
aidanjdonohuecommented, Dec 9, 2021

@loodvn if you’re using PyTorch, using the profiler is probably your best bet for debugging GPU memory issues but I can file a ticket for adding some of these more basic tools to the UI

1reaction
vanpeltcommented, Jul 2, 2020

Hey @coolalexzb we’re currently not exposing these dynamic settings in the UI. The relevant code is here: https://github.com/wandb/client/blob/master/wandb/stats.py#L81 As a quick workaround you could override this method. In the next release of our python library we plan to expose this in the UI, this release is planned for late July.

Read more comments on GitHub >

github_iconTop Results From Across the Web

System metrics: customize logging frequency #1133 - GitHub
Hi there! From the documentation, I find that default metrics will be averaged every 30 seconds. I want to ask if we have...
Read more >
Generate Metrics From Your Logs to View Historical Trends ...
You can create a log-based metric from your log analytics queries by selecting the Generate new Metric option from your graph. Generate a...
Read more >
Log-based metrics overview - Google Cloud
You can create user-defined log-based metrics to track other metrics that are important for your project. For example, you might create a log-based...
Read more >
Chapter 7. Logging performance data with pmlogger Red Hat ...
Record metrics which are not listed in the automatic configuration. Choose custom logging frequencies. Add PMDA with the application metrics. Prerequisites.
Read more >
Logging — PyTorch Lightning 1.8.5.post0 documentation
Track your parameters, metrics, source code and more using Comet. ... You can also pass a custom Logger to the Trainer . ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found