Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[solved] IOU metric (IouCallback) is bigger than 1

See original GitHub issue

I have been using IOUcallback with default parameter to train the Unet model:

runner.train(
    model=model, 
    main_metric = 'iou',
    minimize_metric = False,
    criterion=criterion,
    optimizer=optimizer,
    loaders=loaders,
    logdir=logdir,
    scheduler = scheduler,
    callbacks=[ 
        IouCallback(),
    ],
    num_epochs=num_epochs,
    verbose=True
)

As result metric log looks this way:

[2019-05-26 13:25:18,540] 
0/10 * Epoch 0 (train): _base/lr=0.0010 | _base/momentum=0.9000 | _timers/_fps=14.2914 | _timers/batch_time=0.4415 | _timers/data_time=0.3346 | _timers/model_time=0.1068 | iou=0.2843 | loss=1.2827
0/10 * Epoch 0 (valid): _base/lr=0.0010 | _base/momentum=0.9000 | _timers/_fps=49.2591 | _timers/batch_time=0.1080 | _timers/data_time=0.0271 | _timers/model_time=0.0809 | iou=0.6478 | loss=0.5063
[2019-05-26 13:25:50,207] 
1/10 * Epoch 1 (train): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=12.3666 | _timers/batch_time=0.3442 | _timers/data_time=0.3376 | _timers/model_time=0.0065 | iou=0.5010 | loss=0.4697
1/10 * Epoch 1 (valid): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=46.6509 | _timers/batch_time=0.0335 | _timers/data_time=0.0289 | _timers/model_time=0.0045 | iou=0.9374 | loss=-1.6844
[2019-05-26 13:26:21,913] 
2/10 * Epoch 2 (train): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=12.1218 | _timers/batch_time=0.3469 | _timers/data_time=0.3409 | _timers/model_time=0.0059 | iou=0.5980 | loss=0.0623
2/10 * Epoch 2 (valid): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=44.7021 | _timers/batch_time=0.0343 | _timers/data_time=0.0293 | _timers/model_time=0.0050 | iou=1.0979 | loss=-2.1567
[2019-05-26 13:26:52,914] 
3/10 * Epoch 3 (train): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=12.3602 | _timers/batch_time=0.3504 | _timers/data_time=0.3443 | _timers/model_time=0.0061 | iou=0.5408 | loss=0.0550
3/10 * Epoch 3 (valid): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=48.0609 | _timers/batch_time=0.0321 | _timers/data_time=0.0276 | _timers/model_time=0.0044 | iou=0.9644 | loss=-1.6202
[2019-05-26 13:27:24,687] 
4/10 * Epoch 4 (train): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=12.0270 | _timers/batch_time=0.3544 | _timers/data_time=0.3484 | _timers/model_time=0.0059 | iou=0.7157 | loss=-0.8354
4/10 * Epoch 4 (valid): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=45.5536 | _timers/batch_time=0.0348 | _timers/data_time=0.0301 | _timers/model_time=0.0046 | iou=0.9381 | loss=-1.9926
[2019-05-26 13:27:57,148] 
5/10 * Epoch 5 (train): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=11.9987 | _timers/batch_time=0.3576 | _timers/data_time=0.3516 | _timers/model_time=0.0059 | iou=0.6653 | loss=-0.7207
5/10 * Epoch 5 (valid): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=47.2183 | _timers/batch_time=0.0330 | _timers/data_time=0.0285 | _timers/model_time=0.0044 | iou=1.0183 | loss=-2.6983
[2019-05-26 13:28:28,836] 
6/10 * Epoch 6 (train): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=12.2940 | _timers/batch_time=0.3451 | _timers/data_time=0.3389 | _timers/model_time=0.0061 | iou=0.7951 | loss=-1.6150
6/10 * Epoch 6 (valid): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=45.1915 | _timers/batch_time=0.0339 | _timers/data_time=0.0290 | _timers/model_time=0.0048 | iou=1.3360 | loss=-5.5529
[2019-05-26 13:28:59,628] 
7/10 * Epoch 7 (train): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=12.3576 | _timers/batch_time=0.3424 | _timers/data_time=0.3360 | _timers/model_time=0.0063 | iou=0.7155 | loss=-1.0354
7/10 * Epoch 7 (valid): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=44.9911 | _timers/batch_time=0.0338 | _timers/data_time=0.0288 | _timers/model_time=0.0049 | iou=1.0820 | loss=-1.8514
[2019-05-26 13:29:30,871] 
8/10 * Epoch 8 (train): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=12.5222 | _timers/batch_time=0.3394 | _timers/data_time=0.3337 | _timers/model_time=0.0057 | iou=0.6650 | loss=-0.8128
8/10 * Epoch 8 (valid): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=49.4219 | _timers/batch_time=0.0318 | _timers/data_time=0.0272 | _timers/model_time=0.0046 | iou=1.2081 | loss=-6.1373
[2019-05-26 13:30:01,903] 
9/10 * Epoch 9 (train): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=12.5503 | _timers/batch_time=0.3436 | _timers/data_time=0.3380 | _timers/model_time=0.0056 | iou=0.7887 | loss=-1.5298
9/10 * Epoch 9 (valid): _base/lr=0.0001 | _base/momentum=0.9000 | _timers/_fps=45.4443 | _timers/batch_time=0.0339 | _timers/data_time=0.0291 | _timers/model_time=0.0047 | iou=1.2653 | loss=-3.6054

As you can see some of them higher than 1, is it sum of IOU or I’m doing something wrong?

Issue Analytics

State:
Created 4 years ago
Comments:5 (3 by maintainers)

Top GitHub Comments

2reactions

Scitatorcommented, May 30, 2019

tl;dr solution iou(gt_mask_, pr_mask_, activation=None) all catalyst metrics are logits-based, and iou have sigmoid activation applied for more, please check the iou docs

PS. nevertheless, your contribution for iou tests is always welcome 😃

1reaction

Diyagocommented, Jun 1, 2019

I can offer you assert IOU <= 1 😄

Top Results From Across the Web

Metrics — Catalyst 22.04 documentation

Logic class for all region based metrics, like IoU, Dice, Trevsky. Parameters. metric_fn – metric function, that get statistics and return score. metric_name...

tf.keras.metrics.IoU | TensorFlow v2.11.0

Merges the state from one or more metrics. This method can be used by distributed systems to merge the state computed by different...

Intersection over union (IoU) calculation for evaluating an ...

Intersection over union (IoU) is known to be a good metric for measuring overlap between two bounding boxes or masks.

Catalyst: Accelerated Deep Learning R&D - Morioh

You get a training loop with metrics, early-stopping, model checkpointing, ... a notable use-case/tutorial, or a Kaggle competition solution, or if your ...

Intersection over Union (IoU) - Hasty.ai

Explaining the Intersection over Union metric in detail. ... Additionally, for your information, IoU is referred to as the Jaccard index or Jaccard ......