question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

IoU should ignore void class

See original GitHub issue

Hello, I think the provided code in evaluate.py does not ignore the void label in ground truth masks (which is mapped to the background in the 1D labels); therefore, the performance is underestimated compared to the results of VOC devkit. Here is the link to the line ignoring void pixels in the VOC devkit.

Issue Analytics

  • State:closed
  • Created 7 years ago
  • Reactions:1
  • Comments:7 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
DrSleepcommented, Jan 19, 2017

I can confirm that with weights set the results are improved: from 76.8 mIoU to 80.1 mIoU on the validation set. I have used the code below, and will update the evaluation script accordingly. Thank you for letting know about the issue!

pred = tf.reshape(pred, [-1,])
gt = tf.reshape(label_batch, [-1,])
weights = tf.cast(tf.less_equal(gt, 20), tf.int32)

# mIoU
mIoU, update_op = tf.contrib.metrics.streaming_mean_iou(pred, gt, num_classes=21, weights=weights)
0reactions
DrSleepcommented, Jan 29, 2017

I have pushed the changes to the master branch. Thanks again!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Image segmentation metrics - Keras
The ID of a class to be ignored during metric computation. This is useful, for example, in segmentation problems featuring a "void" class...
Read more >
How to to drop a specific labeled pixels in semantic ...
"Ignore" label is not a label that your model can predict - it is "outside" its range. This label only exists in the...
Read more >
Tutorial 84 - Intersection over Union (IoU) for ... - YouTube
Code associated with these tutorials can be downloaded from here: https://github.com/bnsreenu/python_for_image_processing_APEERThis video ...
Read more >
MIoU Calculation. Computation of MIoU for Multiple-Class…
'Nanmean' is preferred than ordinary mean to ignore the cases where individual IoU value may turn out to be 'nan' because of the...
Read more >
Loss Function Library - Keras & PyTorch | Kaggle
The equation for multi-class BCE by itself will be familiar to anyone who has ... The IoU metric, or Jaccard Index, is similar...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found