Support key points in COCO/TFRecord/YOLO formats

I tried to generate “dump my annotation” to different formats, but only CVAT XML generated properly. I correctly annotated the images, because the CVAT XML contains my points data, and I see the annotation even if I close and reopen the job. YOLO format text files are empty. At TF Record format, the the label_map.pbtxt is generated properly, but the binary .tfrecord file size is 0 byte.

What did I do wrong?

An empty PASCAL VOC file:

<annotation>
    <folder></folder>
    <filename>IMG_160729_071349_0000_RGB.JPG</filename>
    <path></path>
    <source>
        <database>Unknown</database>
    </source>
    <size>
        <width>4608</width>
        <height>3456</height>
        <depth>3</depth>
    </size>
    <segmented>0</segmented>

</annotation>

Part of the CVAT XML file which contains the points:

<image id="0" name="IMG_160729_071349_0000_RGB.JPG" width="4608" height="3456">
    <polygon label="tree" occluded="0" points="810.72,479.23;720.24,541.51; ...
    </polygon>
    <polygon label="tree" occluded="0" points="1791.47,904.28;1802.24,861.19; ...
	</polygon>
    <polygon label="tree" occluded="0" points="1650.91,1209.43;1619.18,1221.18; ...
	</polygon>
    <polygon label="tree" occluded="0" points="782.52,465.13;757.85,451.03;737.87,431.06; ...
	</polygon>
    <polygon label="tree" occluded="0" points="1173.69,992.94;1204.97,995.66;1217.21,965.05; ...
	</polygon>
    <polygon label="tree" occluded="0" points="1631.40,518.13;1593.10,525.65;1593.10,525.65; ...
	</polygon>
    <polygon label="tree" occluded="0" points="1932.48,627.94;1977.53,644.59;2012.78,612.27; ...
	</polygon>
    <polygon label="tree" occluded="0" points="695.08,120.67;616.20,152.63;599.20,160.11; ...
	</polygon>
    <polygon label="roof" occluded="0" points="1937.74,955.25;1880.60,938.72;1829.60,937.78; ...
	</polygon>
  </image>
  <image id="1" name="IMG_160729_071351_0001_RGB.JPG" width="4608" height="3456"> ...
   ...

[azhavoro]

Hi @bessszilard For now only CVAT XML format supports points. https://github.com/opencv/cvat/blob/develop/cvat/apps/annotation/README.md#format-specifications

[zhiltsov-max]

In general, TFrecords have no fixed format. Technically, we could come up with our own format, allowing exporting all CVAT data, but this format would be incompatible with any existing tools and scripts. If there is an established format for keypoints in TFrecords, we can implement it - please, give a link to its specification.

To export lists of points in COCO format, do:

1. Export task in Datumaro format
2. Follow README.md
3. datum project export -f coco_instances [-- --save-images]

This way, you can export points, but they won’t contain visibility info.