Trying to train with HTC, 2 bugs found

Describe the bug

First I get bug #977 Not clear why that line changes the file name. If removed, then I get this exception:

Traceback (most recent call last):
  File "tools/train.py", line 108, in <module>
    main()
  File "tools/train.py", line 104, in main
    logger=logger)
  File "/home/eduardo/dev/mmdetection/mmdet/apis/train.py", line 60, in train_detector
    _non_dist_train(model, dataset, cfg, validate=validate)
  File "/home/eduardo/dev/mmdetection/mmdet/apis/train.py", line 221, in _non_dist_train
    runner.run(data_loaders, cfg.workflow, cfg.total_epochs)
  File "/home/eduardo/miniconda2/envs/pytorch/lib/python3.7/site-packages/mmcv-0.2.12-py3.7-linux-x86_64.egg/mmcv/runner/runner.py", line 358, in run
    epoch_runner(data_loaders[i], **kwargs)
  File "/home/eduardo/miniconda2/envs/pytorch/lib/python3.7/site-packages/mmcv-0.2.12-py3.7-linux-x86_64.egg/mmcv/runner/runner.py", line 264, in train
    self.model, data_batch, train_mode=True, **kwargs)
  File "/home/eduardo/dev/mmdetection/mmdet/apis/train.py", line 38, in batch_processor
    losses = model(**data)
  File "/home/eduardo/miniconda2/envs/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/eduardo/miniconda2/envs/pytorch/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 150, in forward
    return self.module(*inputs[0], **kwargs[0])
  File "/home/eduardo/miniconda2/envs/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/eduardo/dev/mmdetection/mmdet/core/fp16/decorators.py", line 49, in new_func
    return old_func(*args, **kwargs)
  File "/home/eduardo/dev/mmdetection/mmdet/models/detectors/base.py", line 86, in forward
    return self.forward_train(img, img_meta, **kwargs)
  File "/home/eduardo/dev/mmdetection/mmdet/models/detectors/htc.py", line 189, in forward_train
    loss_seg = self.semantic_head.loss(semantic_pred, gt_semantic_seg)
  File "/home/eduardo/dev/mmdetection/mmdet/core/fp16/decorators.py", line 127, in new_func
    return old_func(*args, **kwargs)
  File "/home/eduardo/dev/mmdetection/mmdet/models/mask_heads/fused_semantic_head.py", line 104, in loss
    loss_semantic_seg = self.criterion(mask_pred, labels)
  File "/home/eduardo/miniconda2/envs/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/eduardo/miniconda2/envs/pytorch/lib/python3.7/site-packages/torch/nn/modules/loss.py", line 916, in forward
    ignore_index=self.ignore_index, reduction=self.reduction)
  File "/home/eduardo/miniconda2/envs/pytorch/lib/python3.7/site-packages/torch/nn/functional.py", line 1995, in cross_entropy
    return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
  File "/home/eduardo/miniconda2/envs/pytorch/lib/python3.7/site-packages/torch/nn/functional.py", line 1826, in nll_loss
    ret = torch._C._nn.nll_loss2d(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
RuntimeError: 1only batches of spatial targets supported (non-empty 3D tensors) but got targets of size: : [1, 100, 128, 3]

Reproduction

1. What command or script did you run?

python tools/train.py cfg/htc_x101_32x4d_fpn_20e.py

1. Did you make any modifications on the code or config? Did you understand what you have modified?

Only pointed the configuration to my dataset paths.

1. What dataset did you use?

I am using my own COCO formatted dataset. It worked fine for Retinanet but does not work with HTC (config htc_x101_32x4d_fpn_20e).

With Retinanet I didn’t provide segmentation. Here it seems mandatory (crashes if not provided), so I fill it with 4 coordinates of the bbox rectangle.

Environment

• OS: Ubuntu 18.04.2
• GCC 7.4.0
• PyTorch version 1.2.0 (conda)
• GPU model: 2080 ti
• CUDA 10 and CUDNN 7.6.0

[FantasyJXF]

The error comes from seg map label files, the .png label files should be read with PIL.Image, just like the VOC dataset, these label images are in palette mode. If we just read it with opencv or mmcv, we will get labels shape like H,W,3 which should be H,W, so the torch.nn.CrossEntropy loss function went into error.

[mosheliv]

As this is the third time I see this question, perhaps it should be in the readme? A better error message will also be good.

On Fri, Aug 9, 2019, 18:09 Kai Chen [e-mail] wrote:

HTC is for instance segmentation. Similar to Mask R-CNN, the mask of each instance should be annotated. If your dataset does not provide such annotation, HTC/Mask R-CNN is not proper for usage. Furthermore, HTC can also utilize the semantic segmentation annotation, in the config directory, there are two kind of settings that with or without semantic segmentation annotations.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/open-mmlab/mmdetection/issues/1154?email_source=notifications&email_token=AC7IWC5LKFUG3CG2A6KH2SDQDUCYXA5CNFSM4IKNZPMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD35WIGI#issuecomment-519791641, or mute the thread https://github.com/notifications/unsubscribe-auth/AC7IWC3HIM3QTUC5DVBIMLLQDUCYXANCNFSM4IKNZPMA .