!_src.empty() in function 'cv::cvtColor'

Describe the bug During training at model.fit(dataset) OpenCV gives an error cv2.error: OpenCV(4.3.0) C:\projects\opencv-python\opencv\modules\imgproc\src\color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cv::cvtColor'

Code and Data

from detecto import core, utils, visualize
import matplotlib.pyplot as plt
import os

IMAGE_PATH = os.path.abspath(os.path.join(os.path.dirname( __file__ ), 'images'))
LABEL_PATH = os.path.abspath(os.path.join(os.path.dirname( __file__ ), 'labels'))


'''image = utils.read_image(os.path.join(IMAGE_PATH, 'image0.jpg'))
plt.imshow(image)
plt.show()'''



# Images and XML files in separate folders
dataset = core.Dataset(LABEL_PATH, IMAGE_PATH)

image, target = dataset[0]
print(image, target)

model = core.Model(['bat', 'batter', 'pitch', 'field', 'player', 'scoreboard', 'stumps'])

model.fit(dataset)


# Specify the path to your image
image = utils.read_image(os.path.join(IMAGE_PATH, 'image-3361.jpg'))
predictions = model.predict(image)

# predictions format: (labels, boxes, scores)
labels, boxes, scores = predictions

print(labels) 
print(boxes)
print(scores)

It prints the image and target so we know it’s reading in the dataset:

tensor([[[-0.9705, -0.6794, -0.5596,  ..., -0.3541, -0.9534, -2.1008],
         [-0.9705, -0.6794, -0.5596,  ..., -0.3712, -0.9534, -2.1008],
         [-0.9705, -0.6794, -0.5596,  ..., -0.3541, -0.9363, -2.1008],
         ...,
         [-0.9192, -0.3369, -0.5253,  ..., -0.0801, -0.7650, -2.1008],
         [-0.9192, -0.3369, -0.5253,  ..., -0.0801, -0.7650, -2.1008],
         [-0.9192, -0.3369, -0.5253,  ..., -0.0801, -0.7650, -2.1008]],

        [[-0.3725, -0.0749,  0.0476,  ...,  0.1877, -0.4601, -1.6506],
         [-0.3725, -0.0749,  0.0476,  ...,  0.1702, -0.4601, -1.6506],
         [-0.3725, -0.0749,  0.0476,  ...,  0.1877, -0.4426, -1.6506],
         ...,
         [-0.3375,  0.2577,  0.0651,  ...,  0.3627, -0.3725, -1.7731],
         [-0.3375,  0.2577,  0.0651,  ...,  0.3627, -0.3725, -1.7731],
         [-0.3375,  0.2577,  0.0651,  ...,  0.3627, -0.3725, -1.7731]],

        [[-1.2119, -0.9156, -0.7936,  ..., -0.4275, -0.8633, -1.8044],
         [-1.2119, -0.9156, -0.7936,  ..., -0.4450, -0.8633, -1.8044],
         [-1.2119, -0.9156, -0.7936,  ..., -0.4275, -0.8458, -1.8044],
         ...,
         [-1.3513, -0.7587, -0.9504,  ..., -0.3230, -0.7761, -1.8044],
         [-1.3513, -0.7587, -0.9504,  ..., -0.3230, -0.7761, -1.8044],
         [-1.3513, -0.7587, -0.9504,  ..., -0.3230, -0.7761, -1.8044]]]) {'boxes': tensor([[795, 385, 835, 462]]), 'labels': 'bat'}

Stacktrace

Traceback (most recent call last):
  File "potato.py", line 23, in <module>
    model.fit(dataset)
  File "C:\Users\julis\AppData\Local\Programs\Python\Python38\lib\site-packages\detecto\core.py", line 468, in fit
    for images, targets in dataset:
  File "C:\Users\julis\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\utils\data\dataloader.py", line 363, in __next__
    data = self._next_data()
  File "C:\Users\julis\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\utils\data\dataloader.py", line 403, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "C:\Users\julis\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\utils\data\_utils\fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "C:\Users\julis\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\utils\data\_utils\fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "C:\Users\julis\AppData\Local\Programs\Python\Python38\lib\site-packages\detecto\core.py", line 149, in __getitem__
    image = read_image(img_name)
  File "C:\Users\julis\AppData\Local\Programs\Python\Python38\lib\site-packages\detecto\utils.py", line 134, in read_image
    return cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
cv2.error: OpenCV(4.3.0) C:\projects\opencv-python\opencv\modules\imgproc\src\color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cv::cvtColor'

After some research this is when OpenCV can’t find the image but I don’t understand how this is possible since it prints the first item in the dataset list

Environment:

• OS: Windows 10
• Python version: 3.8
• Detecto version: latest
• torch version: 1.5.0
• torchvision version: 0.6.0

Additional context Image name is : ‘image-3361.jpg’ Label name is: ‘image-3361.xml’

[alankbi]

I think just deleting image-1256.xml should be able to solve it. Basically, when you pass in the XML files to the dataset, it constructs a dataframe that references all of the info stored in those XML files. When you access something like dataset[80], it’ll access row 80 in that dataframe (in other words, the information from your 80th XML file), which includes a path to the associated image, and tries to return that image as well as the info about labels/bounding boxes. However, since that 80th XML file has a path to a nonexistent image, you end up getting that cv2 error when the dataset tries to read it. Therefore, once you remove or update the incorrect XML file, your dataset should go back to working as normal.

[terpdaderp]

I have this issue as well attempted to downgrade OpenCV but to no avail.