Migrate metrics used in all examples from Datasets to Evaluate
See original GitHub issueThe metrics are slowly leaving Datasets (they are being deprecated as we speak) to move to the Evaluate library. We are looking for contributors to help us with the move.
Normally, the migration should be as easy as replacing the import of load_metric
from Datasets to the load
function in Evaluate. See a use in this Accelerate example. To fix all tests, a dependency to evaluate will need to be added in the requirements file (this is the link for PyTorch, there is another one for the Flax examples).
If you’re interested in contributing, please reply to this issue with the examples you plan to move.
Issue Analytics
- State:
- Created a year ago
- Comments:10 (10 by maintainers)
Top Results From Across the Web
Evaluate predictions - Hugging Face
Datasets provides various common and NLP-specific metrics for you to measure your models performance. In this section of the tutorials, you will load...
Read more >Migrate metrics and optimizers | TensorFlow Core
metrics contains all the metric functions and objects. The Metric object can be used with tf.keras.Model and tf.keras.layers.layer to calculate metric values ...
Read more >TPR and FPR or precision or recall. - Towards Data Science
What metrics should be used for evaluating a model on an imbalanced data set? (precision + recall or ROC=TPR+FPR). I always thought the...
Read more >Adding calculated fields - Amazon QuickSight
The value of that aggregated field is recalculated appropriately for each level. This type of aggregation isn't possible during dataset preparation. For example...
Read more >Evaluating models | AutoML Tables - Google Cloud
This page describes how to use evaluation metrics for your model after it is ... are the expected value of each metric on...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Hi, I would like to try to move Pytorch examples.
@sgugger , I would like to work on the tensorflow examples.