bounding box conversions fail with images of different sizes

internally b/242193874

repro code:

images = tf.ragged.constant([np.ones(shape=[20,30,3]), # 2 images
                             np.ones(shape=[25,15,3])], ragged_rank=2)
print("Images shape:", images.shape)
boxes = tf.ragged.constant([[[3,5,15,25], [10,2,12,4], [1,9,7,19]], # 3 boxes
                            [[1,10,24,14], [5,6,14,16]]], ragged_rank=1) # 2 boxes
print("Boxes shape:", boxes.shape)
# boxes originally in "xyxy" format, converting to "rel_xyxy"
kcv.bounding_box.convert_format(boxes, source="xywh", target="rel_xyxy", images=images)

output is :

Images shape: (2, None, None, 3)
Boxes shape: (2, None, 4)

but the bbox conversion fails with error:

ValueError: Index 1 is not uniform

Looking at the code, a single width and a single height are extracted from the parameter “images” which is wrong. Multiple images can have multiple sizes:

def _xyxy_to_rel_xyxy(boxes, images=None):
    if images is None:
        raise RequiresImagesException()
    shape = tf.shape(images)
==> height, width = shape[1], shape[2]
    height, width = tf.cast(height, boxes.dtype), tf.cast(width, boxes.dtype)
    left, top, right, bottom, rest = tf.split(
        boxes, [1, 1, 1, 1, boxes.shape[-1] - 4], axis=-1
    )
    left, right = left / width, right / width
    top, bottom = top / height, bottom / height
    return tf.concat(
        [left, top, right, bottom, rest],
        axis=-1,
    )

[martin-gorner]

I tested. The fix looks good to me. Thanks!

[martin-gorner]

Absolutely still interested. Thank you for your contributions @bhack ! I’ll have the team look at your PR asap.