Azure Custom Vision - Image classified differently by a model during training/testing.
See original GitHub issue- Package Name: azure-cognitiveservices-vision-customvision
- Package Version: azure-cognitiveservices-vision-customvision==3.1.0 azure-common==1.1.28 azure-core==1.23.1 azure-storage-blob==12.11.0
- Operating System: MacOs Monterrey 12.3.1
- Python Version: Python 3.8.13
Describe the bug The same image is classified as tag A during training but then classified as tag B if it’s tested using the quick test image functionality. All of this is calculated at 50% probability threshold.
For the image in question, during training I got:
tag A 99.9%
tag B 0%
but during testing, using the quick test image option, the very same image gets:
tag A 45.7%
tag B 54.2%
that is contradictory.
This means that the stats shown by the GUI and the ones calculated using the SDK/API differ. ( I wouldn’t test every single image used during training through the GUI, because I’d never finish)
API/SDK GUI
precision 100% 100%
recall 99.4% 100%
To Reproduce I’m afraid that this is very model-dependent.
Expected behavior An image should receive the same tag by the model in both training and prediction.
Issue Analytics
- State:
- Created a year ago
- Comments:5 (1 by maintainers)
Top Results From Across the Web
Build an image classification model with the Custom Vision ...
In this quickstart, you'll learn how to use the Custom Vision web portal to create, train, and test an image classification model.
Read more >Image Classification with Azure Custom Vision Service - Medium
In this article, we will see how to use Azure Custom Vision Service to perform an image classification task. Today, we are using...
Read more >Model Evaluation - Implementing Image Classification by ...
Model Evaluation - Implementing Image Classification by Using the Azure Custom Vision Service course from Cloud Academy. Start learning today with our ...
Read more >Building Custom Image Classifier using Azure Custom Vision
... image tagging/labelling, training classification model, evaluating the model, and testing the predictions using Prediction API.
Read more >Create a model with Azure Custom Vision and Python
Our model is going to be a classification model, meaning the model will look at the picture and scores the pictures against the...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Apologies. This ticket was because of an error from my side. I’ll close it.
much appreciated… i don’t really know where else to post this~