question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Azure Custom Vision - Image classified differently by a model during training/testing.

See original GitHub issue
  • Package Name: azure-cognitiveservices-vision-customvision
  • Package Version: azure-cognitiveservices-vision-customvision==3.1.0 azure-common==1.1.28 azure-core==1.23.1 azure-storage-blob==12.11.0
  • Operating System: MacOs Monterrey 12.3.1
  • Python Version: Python 3.8.13

Describe the bug The same image is classified as tag A during training but then classified as tag B if it’s tested using the quick test image functionality. All of this is calculated at 50% probability threshold.

For the image in question, during training I got:

tag A 99.9%
tag B 0%

but during testing, using the quick test image option, the very same image gets:

tag A 45.7%
tag B 54.2%

that is contradictory.

This means that the stats shown by the GUI and the ones calculated using the SDK/API differ. ( I wouldn’t test every single image used during training through the GUI, because I’d never finish)

             API/SDK       GUI
precision    100%          100%
recall       99.4%         100%

To Reproduce I’m afraid that this is very model-dependent.

Expected behavior An image should receive the same tag by the model in both training and prediction.

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:5 (1 by maintainers)

github_iconTop GitHub Comments

1reaction
agonzalez-mobcommented, May 3, 2022

Apologies. This ticket was because of an error from my side. I’ll close it.

1reaction
agonzalez-mobcommented, May 2, 2022

@agonzalez-mob - Thanks for your feedback! Directing this to the right people.

much appreciated… i don’t really know where else to post this~

Read more comments on GitHub >

github_iconTop Results From Across the Web

Build an image classification model with the Custom Vision ...
In this quickstart, you'll learn how to use the Custom Vision web portal to create, train, and test an image classification model.
Read more >
Image Classification with Azure Custom Vision Service - Medium
In this article, we will see how to use Azure Custom Vision Service to perform an image classification task. Today, we are using...
Read more >
Model Evaluation - Implementing Image Classification by ...
Model Evaluation - Implementing Image Classification by Using the Azure Custom Vision Service course from Cloud Academy. Start learning today with our ...
Read more >
Building Custom Image Classifier using Azure Custom Vision
... image tagging/labelling, training classification model, evaluating the model, and testing the predictions using Prediction API.
Read more >
Create a model with Azure Custom Vision and Python
Our model is going to be a classification model, meaning the model will look at the picture and scores the pictures against the...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found