The RLE or Polygon format of "segmentation“ for extending to coco dataset

Hi Detectron,

Recently I tried to add my custom coco data to run Detectron and encountered the following issues. (1) “segmentation” in coco data like below，

{“segmentation”: [[499.71, 397.28,…342.71, 172.31]], “area”: 43466.12825, “iscrowd”: 0, “image_id”: 182155, “bbox”: [338.89, 51.69, 205.82, 367.61], “category_id”: 1, “id”: 1248258},

{“segmentation”: {“counts”: [66916, 6, 587,… 1, 114303], “size”: [594, 640]}, “area”: 6197, “iscrowd”: 1, “image_id”: 284445, “bbox”: [112, 322, 335, 94], “category_id”: 1, “id”: 9.001002844e+11}, The first format of “segmentation” is polygon and the second is need to encode/decode for RLE format.

Above formats can run on Detectron.

(2) I added a new category , and generated a new RLE format for “segmentation” field via coco api encode()/decode() mask. I generated data like this.

“segmentation”: [{"counts": “mng=1fb02O1O1O001N2O001O1O0O2O1O1O001N2O001O1O0O2O1O001O1O1O010000O01000O010000O01000O01000O01000O01N2N2M2O2N2N1O2N2O001O10O?B000O10O1O001^OQ^O9Pb0EQ^O;Wb0OO01O1O1O001O1N2N`jT3”,“size”: [600,1000]}]

I found the bolded characters is different from the original coco “segmentation” json format although it can run on MatterPort’s implementation to Mask-RCNN.

Also, I tried to modify some Detectron’s code to meet my requirement, but very difficult to me because lots of code need to change.

Could you give me some suggestions to run my custom data?

Thanks.

[Sundrops]

@topcomma Maybe you can try to convert mask to polys.

labels_info = []
for mask in mask_list:
    # opencv 3.2
    mask_new, contours, hierarchy = cv2.findContours((mask).astype(np.uint8), cv2.RETR_TREE,
                                                        cv2.CHAIN_APPROX_SIMPLE)
    # before opencv 3.2
    # contours, hierarchy = cv2.findContours((mask).astype(np.uint8), cv2.RETR_TREE,
    #                                                    cv2.CHAIN_APPROX_SIMPLE)
    segmentation = []

    for contour in contours:
        contour = contour.flatten().tolist()
        # segmentation.append(contour)
        if len(contour) > 4:
            segmentation.append(contour)
    if len(segmentation) == 0:
        continue
    # get area, bbox, category_id and so on
    labels_info.append(
        {
            "segmentation": segmentation,  # poly
            "area": area,  # segmentation area
            "iscrowd": 0,
            "image_id": index,
            "bbox": [x1, y1, bbox_w, bbox_h],
            "category_id": category_id,
            "id": label_id
        },
    )

[Sundrops]

@topcomma COCO annotations have two types of segmentation annotations

1. polygon (object instance) [[499.71, 397.28,…342.71, 172.31]] vertexes
2. uncompressed RLE (crowd). “segmentation”: {“counts”: [66916, 6, 587,… 1, 114303], “size”: [594, 640]}, 66916 represents the number of label 0.

The polygon and uncompressed RLE will be converted to compact RLE format withe the MaskApi. The compact RLE format: segmentation": [{“counts”: “mng=1fb02O1O1O001N2O001O1O0O2O1O1O001N2O001O1O0O2O1O001O1O1O010000O01000O010000O01000O01000O01000O01N2N2M2O2N2N1O2N2O001O10O?B000O10O1O001^OQ^O9Pb0EQ^O;Wb0OO01O1O1O001O1N2N`jT3”,“size”: [600,1000]}]