[solved] IOU metric (IouCallback) is bigger than 1
See original GitHub issueI have been using IOUcallback with default parameter to train the Unet model:
runner.train(
model=model,
main_metric = 'iou',
minimize_metric = False,
criterion=criterion,
optimizer=optimizer,
loaders=loaders,
logdir=logdir,
scheduler = scheduler,
callbacks=[
IouCallback(),
],
num_epochs=num_epochs,
verbose=True
)
As result metric log looks this way:
[2019-05-26 13:25:18,540]
0/10 * Epoch 0 (train): _base/lr=0.0010 | _base/momentum=0.9000 | _timers/_fps=14.2914 | _timers/batch_time=0.4415 | _timers/data_time=0.3346 | _timers/model_time=0.1068 | iou=0.2843 | loss=1.2827
0/10 * Epoch 0 (valid): _base/lr=0.0010 | _base/momentum=0.9000 | _timers/_fps=49.2591 | _timers/batch_time=0.1080 | _timers/data_time=0.0271 | _timers/model_time=0.0809 | iou=0.6478 | loss=0.5063
[2019-05-26 13:25:50,207]
1/10 * Epoch 1 (train): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=12.3666 | _timers/batch_time=0.3442 | _timers/data_time=0.3376 | _timers/model_time=0.0065 | iou=0.5010 | loss=0.4697
1/10 * Epoch 1 (valid): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=46.6509 | _timers/batch_time=0.0335 | _timers/data_time=0.0289 | _timers/model_time=0.0045 | iou=0.9374 | loss=-1.6844
[2019-05-26 13:26:21,913]
2/10 * Epoch 2 (train): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=12.1218 | _timers/batch_time=0.3469 | _timers/data_time=0.3409 | _timers/model_time=0.0059 | iou=0.5980 | loss=0.0623
2/10 * Epoch 2 (valid): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=44.7021 | _timers/batch_time=0.0343 | _timers/data_time=0.0293 | _timers/model_time=0.0050 | iou=1.0979 | loss=-2.1567
[2019-05-26 13:26:52,914]
3/10 * Epoch 3 (train): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=12.3602 | _timers/batch_time=0.3504 | _timers/data_time=0.3443 | _timers/model_time=0.0061 | iou=0.5408 | loss=0.0550
3/10 * Epoch 3 (valid): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=48.0609 | _timers/batch_time=0.0321 | _timers/data_time=0.0276 | _timers/model_time=0.0044 | iou=0.9644 | loss=-1.6202
[2019-05-26 13:27:24,687]
4/10 * Epoch 4 (train): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=12.0270 | _timers/batch_time=0.3544 | _timers/data_time=0.3484 | _timers/model_time=0.0059 | iou=0.7157 | loss=-0.8354
4/10 * Epoch 4 (valid): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=45.5536 | _timers/batch_time=0.0348 | _timers/data_time=0.0301 | _timers/model_time=0.0046 | iou=0.9381 | loss=-1.9926
[2019-05-26 13:27:57,148]
5/10 * Epoch 5 (train): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=11.9987 | _timers/batch_time=0.3576 | _timers/data_time=0.3516 | _timers/model_time=0.0059 | iou=0.6653 | loss=-0.7207
5/10 * Epoch 5 (valid): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=47.2183 | _timers/batch_time=0.0330 | _timers/data_time=0.0285 | _timers/model_time=0.0044 | iou=1.0183 | loss=-2.6983
[2019-05-26 13:28:28,836]
6/10 * Epoch 6 (train): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=12.2940 | _timers/batch_time=0.3451 | _timers/data_time=0.3389 | _timers/model_time=0.0061 | iou=0.7951 | loss=-1.6150
6/10 * Epoch 6 (valid): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=45.1915 | _timers/batch_time=0.0339 | _timers/data_time=0.0290 | _timers/model_time=0.0048 | iou=1.3360 | loss=-5.5529
[2019-05-26 13:28:59,628]
7/10 * Epoch 7 (train): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=12.3576 | _timers/batch_time=0.3424 | _timers/data_time=0.3360 | _timers/model_time=0.0063 | iou=0.7155 | loss=-1.0354
7/10 * Epoch 7 (valid): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=44.9911 | _timers/batch_time=0.0338 | _timers/data_time=0.0288 | _timers/model_time=0.0049 | iou=1.0820 | loss=-1.8514
[2019-05-26 13:29:30,871]
8/10 * Epoch 8 (train): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=12.5222 | _timers/batch_time=0.3394 | _timers/data_time=0.3337 | _timers/model_time=0.0057 | iou=0.6650 | loss=-0.8128
8/10 * Epoch 8 (valid): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=49.4219 | _timers/batch_time=0.0318 | _timers/data_time=0.0272 | _timers/model_time=0.0046 | iou=1.2081 | loss=-6.1373
[2019-05-26 13:30:01,903]
9/10 * Epoch 9 (train): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=12.5503 | _timers/batch_time=0.3436 | _timers/data_time=0.3380 | _timers/model_time=0.0056 | iou=0.7887 | loss=-1.5298
9/10 * Epoch 9 (valid): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=45.4443 | _timers/batch_time=0.0339 | _timers/data_time=0.0291 | _timers/model_time=0.0047 | iou=1.2653 | loss=-3.6054
As you can see some of them higher than 1, is it sum of IOU or I’m doing something wrong?
Issue Analytics
- State:
- Created 4 years ago
- Comments:5 (3 by maintainers)
Top Results From Across the Web
Metrics — Catalyst 22.04 documentation
Logic class for all region based metrics, like IoU, Dice, Trevsky. Parameters. metric_fn – metric function, that get statistics and return score. metric_name...
Read more >tf.keras.metrics.IoU | TensorFlow v2.11.0
Merges the state from one or more metrics. This method can be used by distributed systems to merge the state computed by different...
Read more >Intersection over union (IoU) calculation for evaluating an ...
Intersection over union (IoU) is known to be a good metric for measuring overlap between two bounding boxes or masks.
Read more >Catalyst: Accelerated Deep Learning R&D - Morioh
You get a training loop with metrics, early-stopping, model checkpointing, ... a notable use-case/tutorial, or a Kaggle competition solution, or if your ...
Read more >Intersection over Union (IoU) - Hasty.ai
Explaining the Intersection over Union metric in detail. ... Additionally, for your information, IoU is referred to as the Jaccard index or Jaccard ......
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
tl;dr solution
iou(gt_mask_, pr_mask_, activation=None)
all catalyst metrics are logits-based, and iou havesigmoid
activation applied for more, please check the iou docsPS. nevertheless, your contribution for iou tests is always welcome 😃
I can offer you
assert IOU <= 1
😄