Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Throw example for instance segmentation with COCO data format during training evluation

See original GitHub issue

🐛 Bug

Describe the bug A clear and concise description of what the bug is.

https://github.com/airctic/icevision/blob/60b5a6db2e86ebc36d91709b92d06cacd5ccab8d/icevision/models/mmdet/common/mask/prediction.py#L120

The code above throw exception following the instance segmentation tutorial with COCO data format.

The raw_mask in line https://github.com/airctic/icevision/blob/60b5a6db2e86ebc36d91709b92d06cacd5ccab8d/icevision/models/mmdet/common/mask/prediction.py#L113 return an numpy array like this:

[[],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [array([[False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         ...,
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         [False,  True, False, ..., False, False, False]]),
  array([[False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         ...,
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False]]),
  array([[False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         ...,
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False]]),
  array([[False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         ...,
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False]]),
  array([[False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         ...,
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False]]),
  array([[False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         ...,
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False]]),
  array([[False, False,  True, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         ...,
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False]])],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [array([[False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         ...,
         [ True,  True,  True, ..., False, False, False],
         [ True,  True,  True, ..., False, False, False],
         [ True,  True,  True, ..., False, False, False]]),
  array([[False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         ...,
         [ True,  True,  True, ..., False, False, False],
         [ True,  True,  True, ..., False, False, False],
         [ True,  True,  True, ..., False, False, False]]),
  array([[False, False, False, ...,  True,  True,  True],
         [False, False, False, ...,  True,  True,  True],
         [False, False, False, ...,  True,  True,  True],
         ...,
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False]]),
  array([[False, False, False, ...,  True,  True, False],
         [False, False, False, ...,  True,  True,  True],
         [False, False, False, ...,  True,  True,  True],
         ...,
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False]]),
  array([[False,  True,  True, ..., False, False, False],
         [False,  True,  True, ..., False, False, False],
         [False,  True,  True, ..., False, False, False],
         ...,
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False]]),
  array([[ True,  True,  True, ..., False, False, False],
         [ True,  True,  True, ..., False, False, False],
         [ True,  True,  True, ..., False, False, False],
         ...,
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False]]),
  array([[ True,  True,  True, ..., False, False, False],
         [ True,  True,  True, ..., False, False, False],
         [ True,  True,  True, ..., False, False, False],
         ...,
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False]]),
  array([[False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         [False, False, False, ..., False, False, False],
         ...,
         [ True,  True,  True, ..., False, False, False],
         [ True,  True,  True, ..., False, False, False],
         [ True,  True,  True, ..., False, False, False]])],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 [],
 []]

And with the code np.vstack(raw_mask) will throw the exception.

File /usr/local/lib/python3.8/site-packages/icevision/models/mmdet/common/mask/prediction.py:120, in convert_raw_prediction(sample, raw_pred, record, detection_threshold, keep_image)
--> 120 keep_masks = MaskArray(np.vstack(raw_masks)[keep_mask])
    123 keep_labels = convert_background_from_last_to_zero(
    124     label_ids=keep_labels, class_map=record.detection.class_map
    125 )
    127 pred = BaseRecord(
    128     (
    129         ScoresRecordComponent(),
   (...)
    134     )
    135 )

File <__array_function__ internals>:180, in vstack(*args, **kwargs)

File /usr/local/lib/python3.8/site-packages/numpy/core/shape_base.py:282, in vstack(tup)
    280 if not isinstance(arrs, list):
    281     arrs = [arrs]
--> 282 return _nx.concatenate(arrs, 0)

File <__array_function__ internals>:180, in concatenate(*args, **kwargs)

ValueError: all the input arrays must have same number of dimensions, but the array at index 0 has 2 dimension(s) and the array at index 9 has 3 dimension(s)

To Reproduce Steps to reproduce the behavior:

Use the code below:

from icevision.all import *

selection = 1

if selection == 1:
    model_type = models.mmdet.mask_rcnn
    backbone = model_type.backbones.mask_rcnn_swin_t_p4_w7_fpn_1x_coco

parser = parsers.COCOMaskParser(
    img_dir="/input1/train2017/",
    annotations_filepath="instances_minitrain2017.json",
)

train_rs, valid_rs = parser.parse(RandomSplitter([1, 0], seed=42))

image_size = 640
train_tfms = tfms.A.Adapter([*tfms.A.resize_and_pad(image_size), tfms.A.Normalize()])
valid_tfms = tfms.A.Adapter([*tfms.A.resize_and_pad(image_size), tfms.A.Normalize()])

train_ds = Dataset(train_rs, train_tfms)
valid_ds = Dataset(train_rs, valid_tfms)

train_dl = model_type.train_dl(train_ds, batch_size=4, num_workers=8, shuffle=True)
valid_dl = model_type.valid_dl(valid_ds, batch_size=4, num_workers=8, shuffle=False)

model = model_type.model(backbone=backbone(pretrained=True), num_classes=len(parser.class_map)) 

metrics = [COCOMetric(metric_type=COCOMetricType.mask)]

class LightModel(model_type.lightning.ModelAdapter):
    def configure_optimizers(self):
        return Adam(self.parameters(), lr=8e-5)
    
light_model = LightModel(model, metrics=metrics)

trainer = pl.Trainer(max_epochs=5, gpus=1)
trainer.fit(light_model, train_dl, valid_dl)

Expected behavior A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots to help explain your problem.

image

Desktop (please complete the following information):

  • OS: [e.g. ubuntu 18.04]

Additional context Add any other context about the problem here.

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:7 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
fstrothcommented, Apr 24, 2022

I didn’t have time yet to test on LVIS. I will give an update when I have tested it.

0reactions
fstrothcommented, Apr 24, 2022

I get the same error as you with LVIS. Thank you for the PR!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Getting started with COCO dataset - Towards Data Science
We've explored the COCO dataset format for the most popular tasks: object detection, object segmentation and stuff segmentation. Official COCO ...
Read more >
COCO - Common Objects in Context
COCO is a large-scale object detection, segmentation, and captioning dataset. COCO has several features: Object segmentation; Recognition in context; Superpixel ...
Read more >
how object segmentation are written in dataset json files? #111
Hi Guys, so I am wanting to do instance segmentation and the model that I am using expects the data to be in...
Read more >
How to create custom COCO data set for instance segmentation
In this post, I will show you how simple it is to create your custom COCO dataset and train an instance segmentation model...
Read more >
2: Train with customized datasets
Note: MMDetection only supports evaluating mask AP of dataset in COCO format for now. So for instance segmentation task users should convert the...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

UnpicklingError: invalid load key, '<'.

Next page

Add detection_threshold arg for all Lightning adapters