Augment Keypoints not always correct

Hi!

Thanks a lot for the library, it really works well. The only thing I have issues with is the keypoint augmentation, which works sometimes but in other cases it returns a set of negative coordinate values which are then of course not plotted inside the image when I use ‘draw_on_image’ and show it using matplotlib afterwards. I’m currently using this augmentation sequence:

seq = iaa.Sequential([
    iaa.Fliplr(0.5), # horizontally flip 50% of the images
    iaa.Flipud(0.5), # vertically flip 50% of all images
    #iaa.GaussianBlur(sigma=(0, 3.0)), # blur images with a sigma of 0 to 3.0
    iaa.Affine(
            scale={"x": (0.8, 1.2), "y": (0.8, 1.2)}, # scale images to 80-120% of their size, individually per axis
            translate_px={"x": (-16, 16), "y": (-16, 16)}, # translate by -16 to +16 pixels (per axis)
            rotate=(-45, 45), # rotate by -45 to +45 degrees
            shear=(-16, 16), # shear by -16 to +16 degrees
            order=ia.ALL, # use any of scikit-image's interpolation methods
            #cval=(0, 1.0), # if mode is constant, use a cval between 0 and 1.0
            #mode=ia.ALL # use any of scikit-image's warping modes (see 2nd image from the top for examples)
        )
    ],
    random_order=True # do all of the above in random order
)

seq_det = seq.to_deterministic() # call this for each batch again, NOT only once at the start

For example, when I use this input image:

I get results like this:

Does anyone have an idea what could be the problem?

Thanks a lot!

UPDATE: It seems like the keypoint augmentation only works if a single command is used in iaa.Affine() at once. If use several, only the first is use to augment keypoints. What works is to use several iaa.Affine() commands in series, with each containing a different image manipulation, which is fine for now.

[HassanBinHaroon]

Hi aleju, I am trying to augment images with keypoints How can I deal with missing key points. should I pass them in as -1. How can I prevent images/keypoints from going out of the frame while rotating images? Keeping them in the frame and rotating them seq = iaa.Sequential([ iaa.Multiply((0.5, 1.5)), iaa.Sometimes(0.2, iaa.GaussianBlur(sigma=(0, 0.5))), iaa.Fliplr(1.0), iaa.Affine(rotate=(-7,7),)], random_order=True)

One way of fixing it is to apply a check and if the coordinates become greater than the frame dimensions (while rotating) limit them to the maximum value.

[mukeshnarendran7]

Hi aleju, I am trying to augment images with keypoints How can I deal with missing key points. should I pass them in as -1. How can I prevent images/keypoints from going out of the frame while rotating images? Keeping them in the frame and rotating them seq = iaa.Sequential([ iaa.Multiply((0.5, 1.5)), iaa.Sometimes(0.2, iaa.GaussianBlur(sigma=(0, 0.5))), iaa.Fliplr(1.0), iaa.Affine(rotate=(-7,7),)], random_order=True)