question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Migrate metrics used in all examples from Datasets to Evaluate

See original GitHub issue

The metrics are slowly leaving Datasets (they are being deprecated as we speak) to move to the Evaluate library. We are looking for contributors to help us with the move.

Normally, the migration should be as easy as replacing the import of load_metric from Datasets to the load function in Evaluate. See a use in this Accelerate example. To fix all tests, a dependency to evaluate will need to be added in the requirements file (this is the link for PyTorch, there is another one for the Flax examples).

If you’re interested in contributing, please reply to this issue with the examples you plan to move.

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:10 (10 by maintainers)

github_iconTop GitHub Comments

2reactions
atturaioecommented, Jul 27, 2022

Hi, I would like to try to move Pytorch examples.

1reaction
VijayKalmathcommented, Jul 27, 2022

@sgugger , I would like to work on the tensorflow examples.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Evaluate predictions - Hugging Face
Datasets provides various common and NLP-specific metrics for you to measure your models performance. In this section of the tutorials, you will load...
Read more >
Migrate metrics and optimizers | TensorFlow Core
metrics contains all the metric functions and objects. The Metric object can be used with tf.keras.Model and tf.keras.layers.layer to calculate metric values ...
Read more >
TPR and FPR or precision or recall. - Towards Data Science
What metrics should be used for evaluating a model on an imbalanced data set? (precision + recall or ROC=TPR+FPR). I always thought the...
Read more >
Adding calculated fields - Amazon QuickSight
The value of that aggregated field is recalculated appropriately for each level. This type of aggregation isn't possible during dataset preparation. For example...
Read more >
Evaluating models | AutoML Tables - Google Cloud
This page describes how to use evaluation metrics for your model after it is ... are the expected value of each metric on...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found