Got error while using visualizer.draw_dataset_dict()

How To Reproduce the Issue

1. what changes you made (git diff) or what code you wrote

I didn’t make any change, and the code I wrote is the following:

import random
from detectron2.data.datasets import register_coco_instances
from detectron2.utils.visualizer import Visualizer
from detectron2.data import MetadataCatalog, DatasetCatalog

register_coco_instances("femoris_data_rle", {}, "./lung/annotations/total_rle.json", "./lung/tri_img")
femoris_metadata = MetadataCatalog.get("femoris_data_rle")
dataset_dicts = DatasetCatalog.get("femoris_data_rle")

for d in random.sample(dataset_dicts, 3):
    img = cv2.imread(d["file_name"])[:, :, ::-1]
    img_copy = img.copy()
    visualizer = Visualizer(img_copy, metadata=femoris_metadata, scale=0.5)
    vis = visualizer.draw_dataset_dict(d)
    cv2_imshow(vis.get_image()[:, :, ::-1])

The elements in dataset_dicts looks like this:

{'annotations': [{'bbox': [368.9946492271106,
    330.891438763377,
    13.148537455410235,
    13.644708680142685],
   'bbox_mode': <BoxMode.XYWH_ABS: 1>,
   'category_id': 0,
   'iscrowd': 1,
   'segmentation': {'counts': '_jh52m?2N2N2N2O100O10O001N1O2MceP2',
    'size': [512, 512]}}],
 'file_name': './lung/tri_img/44_1_1_1_38.png',
 'height': 512,
 'image_id': 1,
 'width': 512}

I’ve actually ran this code successfully while using polygon format as segmentation, but got this problem as soon as I change my annotations into encoded RLEs.

1. what exact command you run

I just ran the code above in a Jupyter Notebook inside Google Colaboratory.

1. what you observed (including the full logs):

TypeError                                 Traceback (most recent call last)
<ipython-input-65-fe8a14d3a581> in <module>()
      5     img_copy = img.copy()
      6     visualizer = Visualizer(img_copy, metadata=fruits_nuts_metadata, scale=0.5)
----> 7     vis = visualizer.draw_dataset_dict(d)
      8     cv2_imshow(vis.get_image()[:, :, ::-1])

3 frames
/content/gdrive/My Drive/detectron2_repo/detectron2/utils/visualizer.py in draw_dataset_dict(self, dic)
    496                 for i, a in zip(labels, annos)
    497             ]
--> 498             self.overlay_instances(labels=labels, boxes=boxes, masks=masks, keypoints=keypts)
    499 
    500         sem_seg = dic.get("sem_seg", None)

/content/gdrive/My Drive/detectron2_repo/detectron2/utils/visualizer.py in overlay_instances(self, boxes, labels, masks, keypoints, assigned_colors, alpha)
    592 
    593             if masks is not None:
--> 594                 for segment in masks[i].polygons:
    595                     self.draw_polygon(segment.reshape(-1, 2), color, alpha=alpha)
    596 

/content/gdrive/My Drive/detectron2_repo/detectron2/utils/visualizer.py in polygons(self)
     94     def polygons(self):
     95         if self._polygons is None:
---> 96             self._polygons, self._has_holes = self.mask_to_polygons(self._mask)
     97         return self._polygons
     98 

/content/gdrive/My Drive/detectron2_repo/detectron2/utils/visualizer.py in mask_to_polygons(self, mask)
    111         # Internal contours (holes) are placed in hierarchy-2.
    112         # cv2.CHAIN_APPROX_NONE flag gets vertices of polygons from contours.
--> 113         res = cv2.findContours(mask.astype("uint8"), cv2.RETR_CCOMP, cv2.CHAIN_APPROX_NONE)
    114         hierarchy = res[-1]
    115         if hierarchy is None:  # empty mask

TypeError: Layout of the output array image is incompatible with cv::Mat (step[ndims-1] != elemsize or step[1] != elemsize*nchannels)

Environment

sys.platform              linux
Python                    3.6.8 (default, Oct  7 2019, 12:59:55) [GCC 8.3.0]
Numpy                     1.17.3
Detectron2 Compiler       GCC 7.4
Detectron2 CUDA Compiler  10.0
DETECTRON2_ENV_MODULE     <not set>
PyTorch                   1.3.1
PyTorch Debug Build       False
torchvision               0.4.2
CUDA available            True
GPU 0                     Tesla K80
CUDA_HOME                 /usr/local/cuda
NVCC                      Cuda compilation tools, release 10.0, V10.0.130
Pillow                    4.3.0
cv2                       3.4.3
------------------------  --------------------------------------------------
PyTorch built with:
  - GCC 7.3
  - Intel(R) Math Kernel Library Version 2019.0.4 Product Build 20190411 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v0.20.5 (Git Hash 0125f28c61c1f822fd48570b4c1066f96fcb9b2e)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - NNPACK is enabled
  - CUDA Runtime 10.1
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_37,code=compute_37
  - CuDNN 7.6.3
  - Magma 2.5.1
  - Build settings: BLAS=MKL, BUILD_NAMEDTENSOR=OFF, BUILD_TYPE=Release, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -fopenmp -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Wno-stringop-overflow, DISABLE_NUMA=1, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, USE_CUDA=True, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_STATIC_DISPATCH=OFF, 

[ppwwyyxx]

I can now reproduce the issue with opencv 3.4.3. It turns out that old versions of cv2.findContours does not support incontiguous array. We will fix it shortly.

[teddybear0212]

Well I just ran the code you provided above, and the same error occurred! I really don’t know what’s missing. I’ve given you all the code I’ve written so far, so I’m not sure what else can I provide. I’m currently using OpenCV 3.4.3. Maybe that will help?

I ran the following code and the error occurred:

from detectron2.data.datasets import register_coco_instances
from detectron2.utils.visualizer import Visualizer
from detectron2.data import MetadataCatalog, DatasetCatalog
from detectron2.structures import BoxMode
import numpy as np
from google.colab import drive
import cv2
import os

drive.mount('/content/gdrive')
os.chdir('gdrive/My Drive')

register_coco_instances("femoris_data_rle", {}, "./lung/annotations/total_rle.json", "./lung/tri_img")
femoris_metadata = MetadataCatalog.get("femoris_data_rle")
dataset_dicts = DatasetCatalog.get("femoris_data_rle")

img = np.random.rand(512, 512, 3) * 255
dic = {'annotations': [{'bbox': [368.9946492271106,
                                 330.891438763377,
                                 13.148537455410235,
                                 13.644708680142685],
                        'bbox_mode': BoxMode.XYWH_ABS,
                        'category_id': 0,
                        'iscrowd': 1,
                        'segmentation': {'counts': '_jh52m?2N2N2N2O100O10O001N1O2MceP2',
                                         'size': [512, 512]}}],
               'height': 512,
               'image_id': 1,
               'width': 512}
v = Visualizer(img, femoris_metadata)
v.draw_dataset_dict(dic)