question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[examples] replicate the new `--log_level` feature to all trainer-based pytorch examples

See original GitHub issue

https://github.com/huggingface/transformers/pull/12276 introduced a new --log_level feature, which now allows users to set their desired log level via CLI or TrainingArguments.

run_translation.py was used as a “model” for other examples.

Now we need to replicate this to all other Trainer-based examples under examples/pytorch/, the 3 changes are

  1. importing datasets
  2. using training_args.get_node_log_level() and setting log_level in 3 modules
  3. replacing datasets object name with raw_datasets, since otherwise we have a conflict with datasets the module.

and the relevant diff is here

and of course since we don’t quite have extensive tests for examples, you can just test with a staple cmd from the corresponding README.md with --log_level=error and check that almost all logs are gone.

This is open to all.

And thank you.

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:7 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
stas00commented, Jun 22, 2021

@bhadreshpsavani, once https://github.com/huggingface/transformers/pull/12309 gets merged, please rebase your branch and rename

-    get_node_log_level()
+    get_process_log_level()

Thank you!

1reaction
stas00commented, Jun 22, 2021

Yes, of course! Thank you, @bhadreshpsavani

Read more comments on GitHub >

github_iconTop Results From Across the Web

transformers/utils_qa.py at main · huggingface ... - GitHub
logger.info(f"Post-processing {len(examples)} example predictions split into {len(features)} features.") # Let's loop over all the examples! for example_index, ...
Read more >
08. PyTorch Paper Replicating - Zero to Mastery Learn ...
And the goal of paper replicating is to take replicate these advances with code so you can use the techniques for your own...
Read more >
8. Logging in Torchserve - PyTorch
These logs collect all the logs from TorchServe and from the backend workers (the custom model code). ... Some examples of logs are...
Read more >
huggingface/transformers: v4.9.0: TensorFlow examples ...
... [Examples] Replicates the new --log_level feature to all trainer-based pytorch #12359 (@bhadreshpsavani) [Examples] Update Example ...
Read more >
How to add a custom loglevel to Python's logging facility
Fixed example that checks if the logging level is enabled: ... __init__.py for Python 2.7, this is what all the standard log functions...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found