KeyError while running the model on Custom Dataset

Hi! Thanks for this repo. I am trying to run this model on a custom dataset and I face this error. Here is the stack trace:

Traceback (most recent call last):
  File "train.py", line 386, in <module>
    train()
  File "train.py", line 215, in train
    for datum in data_loader:
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 582, in __next__
    return self._process_next_batch(batch)
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 606, in _process_next_batch
    raise Exception("KeyError:" + batch.exc_msg)
Exception: KeyError:Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/worker.py", line 99, in _worker_loop
    samples = collate_fn([dataset[i] for i in batch_indices])
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/worker.py", line 99, in <listcomp>
    samples = collate_fn([dataset[i] for i in batch_indices])
  File "/home/user/yolact/data/coco.py", line 87, in __getitem__
    im, gt, masks, h, w, num_crowds = self.pull_item(index)
  File "/home/user/yolact/data/coco.py", line 126, in pull_item
    file_name = self.coco.loadImgs(img_id)[0]['file_name']
  File "/usr/local/lib/python3.6/dist-packages/pycocotools/coco.py", line 229, in loadImgs
    return [self.imgs[id] for id in ids]
  File "/usr/local/lib/python3.6/dist-packages/pycocotools/coco.py", line 229, in <listcomp>
    return [self.imgs[id] for id in ids]
KeyError: 7

The value changes every time. I have shown an example where the KeyError threw was 7. I followed the answer from issue #40 , but still, the error persists. My class id’s start from 1 and end in 10 (inclusive). Here is my config.py:

my_dataset= dataset_base.copy({
    'name': 'my dataset',
    'train_images': '/home/user/dataset/train/',
    'train_info': '/home/user/dataset/train/via_region_data.json',

    'valid_images':'/home/user/dataset/val/',
    'valid_info': '/home/user/dataset/val/via_region_data.json',

    'has_gt': True,
    'class_names': ('BWK12', 'LWK1', 'LWK2', 'LWK3', 'LWK4', 'LWK5', 'SWK1/2', 'Cage', 'Schraube', 'Stab'),
    'label_map': {1:1, 2:2, 3:3, 4:4, 5:5, 6:6, 7:7, 8:8, 9:9, 10:10}
})

Here is the configuration:

yolact_base_config = coco_base_config.copy({
    'name': 'yolact_base',

    # Dataset stuff
    'dataset': my_dataset,
    'num_classes': len(spine_dataset.class_names)+1,
    .........
)}

[dbolya]

OH, your last observation is actually critical.

Notice something wrong with your annotations? Specifically on this line:

"image_id": "0"

That’s a string not an integer!

It’s executing the array branch because it’s operating on each character of the string, and it can’t find the strings '2' and '7', because of course your IDs are integers! I’m not sure why your first stack trace has KeyError: 7, but your second has KeyError: '2', which indicates that you passed in a string instead of an integer.

So, the solution would be to make sure all the image ids and annotation image_ids are integers (since your first error message had KeyError: 7, it might be that image id 7 has a string for its id field, while all annotations for image id 2 have a string for their image_id fields).

[CodeXiaoLingYun]

@dbolya Thanks for pointing it out. The train.json file was actually exported from VGG Annotation Tool in COCO JSON format and I just appended the categories field to it. My bad that I didn’t notice it. I ran the code after editing, it got past the above error and stopped at a new one. Here is the stacktrace:

Target:  [{'id': 80, 'image_id': 10, 'segmentation': [612, 345, 900, 454, 1018, 459, 926, 704, 695, 616, 529, 590, 581, 542, 612, 463], 'area': 175551, 'bbox': [529, 345, 489, 359], 'iscrowd': 0, 'category_id': 8}]

Traceback (most recent call last):
  File "train.py", line 386, in <module>
    train()
  File "train.py", line 215, in train
    for datum in data_loader:
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 582, in __next__
    return self._process_next_batch(batch)
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 608, in _process_next_batch
    raise batch.exc_type(batch.exc_msg)
TypeError: Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/worker.py", line 99, in _worker_loop
    samples = collate_fn([dataset[i] for i in batch_indices])
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/worker.py", line 99, in <listcomp>
    samples = collate_fn([dataset[i] for i in batch_indices])
  File "/home/user/yolact/data/coco.py", line 87, in __getitem__
    im, gt, masks, h, w, num_crowds = self.pull_item(index)
  File "/home/user/yolact/data/coco.py", line 138, in pull_item
    masks = [self.coco.annToMask(obj).reshape(-1) for obj in target]
  File "/home/user/yolact/data/coco.py", line 138, in <listcomp>
    masks = [self.coco.annToMask(obj).reshape(-1) for obj in target]
  File "/usr/local/lib/python3.6/dist-packages/pycocotools/coco.py", line 431, in annToMask
    rle = self.annToRLE(ann)
  File "/usr/local/lib/python3.6/dist-packages/pycocotools/coco.py", line 416, in annToRLE
    rles = maskUtils.frPyObjects(segm, h, w)
  File "pycocotools/_mask.pyx", line 292, in pycocotools._mask.frPyObjects
TypeError: object of type 'int' has no len()

Tried to figure out the error, but couldn’t.

i also meet the first question. And i change the type to int,it is OK. But i alse meet the second question as you.

File “pycocotools/_mask.pyx”, line 292, in pycocotools._mask.frPyObjects TypeError: object of type ‘int’ has no len()