Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

How to use Metrics class to check the performance of the trained model

See original GitHub issue

📓 New <Tutorial/Example>

Is this a request for a tutorial or for an example?
Tutorial

What is the task? Lets say I trained a model and now I want to check its performance on the brand new dataset. I don’t want to retrain the whole model, I just want to check its performance. How to do it using Metric Class. Lets assume that COCO metric is fine for me.

metric = COCOMetric(metric_type=COCOMetricType.bbox)

How do I use it so I know exact performance on the new dataset?

Don’t remove Main issue for examples: #39

Issue Analytics

State:
Created 3 years ago
Comments:12 (10 by maintainers)

Top GitHub Comments

1reaction

potipotcommented, Dec 19, 2021

I’d say this could be a FAQ so we should think of showcasing this in a notebook linked on a website.

1reaction

FraPochetticommented, Dec 19, 2021

There is an option to test a trained model. Assuming its weights are already loaded you can use for fastai:
learner.validate(test_dl) 
for pytorch-lightning
result = trainer.test(lightning_model, test_dl)

Would this calculate the COCOMetric? I guess it would compute any metric the learner (or PL Module) was instantiated with.

Top Results From Across the Web

Performance Metrics in Machine Learning [Complete Guide]

Performance metrics are a part of every machine learning pipeline. They tell you if you're making progress, and put a number on it....

Metrics to Evaluate your Machine Learning Algorithm

Most of the times we use classification accuracy to measure the performance of our model, however it is not enough to truly judge...

Key Machine Learning Metrics to Evaluate Model Performance

Machine learning metrics help you quantify the performance of a machine learning model once it's already trained. These figures give you an ...

How to Use Metrics for Deep Learning with Keras in Python

This is particularly useful if you want to keep track of a performance measure that better captures the skill of your model during...

Evaluate Models Using Metrics | Machine Learning

If you're predicting a small number of classes, look at per-class metrics individually. When predicting on many classes, you can average the per ......