Potential bug in angle_axis_to_rotation_matrix

Hi !

Thanks for the great work 😃 I am observing some surprising behavior in rotation_matrix_to_angle_axis, and I think it is worth raising although I am not yet sure it is actually a bug or a problem with my interpretation of the functions.

The main observation is that I expect a cycle of conversion to be consistent (e.g. apllying angle_axis_to_rotation_matrix(rotation_matrix_to_axis_angle()) should be close to identity.

However I observe that cycling axis-angles through rotation matrixes produces different results.

import torch
import numpy as np
from scipy.stats import special_ortho_group

from kornia.geometry.conversions import rotation_matrix_to_angle_axis as rm2aa
from kornia.geometry.conversions import angle_axis_to_rotation_matrix as aa2rm
from kornia.geometry.conversions import angle_axis_to_quaternion as aa2quat
from kornia.geometry.conversions import quaternion_to_angle_axis as quat2aa
from kornia.geometry.conversions import quaternion_to_rotation_matrix as quat2rm
from kornia.geometry.conversions import rotation_matrix_to_quaternion as rm2quat

batch_size = 8

rand_aa = torch.rand(batch_size, 3)
# axis_angles --> rotation_matrix --> axis_angle
aa_cycle_through_rm = rm2aa(aa2rm(rand_aa))
aa_through_rm_error = (aa_cycle_through_rm - rand_aa).abs().max(1)[0]
print(f"Reconstruction errors for angle_axis --> rotation_matrix --> angle_axis cycle\n"
        f"errors: {aa_through_rm_error}")  # Large values ~1-2

# axis_angles --> quaternions --> rotation_matrix --> quaternions --> axis_angle
aa_cycle_through_rm_and_quat = quat2aa(rm2quat(quat2rm(aa2quat(rand_aa))))                                                                      aa_through_rm_and_quat_error = (aa_cycle_through_rm_and_quat - rand_aa).abs().max(1)[0]                                                         print(f"Reconstruction errors for angle_axis --> quat --> rotation_matrix --> quat --> angle_axis cycle\n"
         f"errors: {aa_through_rm_and_quat_error}")  # Small values ~1e-8     

Since rotation_matrix_to_angle_axis is actually the composition of rotation_matrix_to_quaternion and quaternion_to_angle_axis (as I observe from https://github.com/kornia/kornia/blob/a6e668519e38a723363ed155819d25d84a068d1c/kornia/geometry/conversions.py#L232) the problem seems to come from angle_axis_to_rotation_matrix.

Further investigations gave the following script :

import torch
import numpy as np
from scipy.stats import special_ortho_group
from kornia.geometry.conversions import rotation_matrix_to_angle_axis as rm2aa
from kornia.geometry.conversions import angle_axis_to_rotation_matrix as aa2rm
from kornia.geometry.conversions import angle_axis_to_quaternion as aa2quat
from kornia.geometry.conversions import quaternion_to_angle_axis as quat2aa
from kornia.geometry.conversions import quaternion_to_rotation_matrix as quat2rm
from kornia.geometry.conversions import rotation_matrix_to_quaternion as rm2quat

# Sample random rotation matrixes
random_rots = []
for batch_idx in range(batch_size):
    random_rots.append(special_ortho_group.rvs(3))
random_rots = torch.Tensor(np.array(random_rots))

# Generate random axis angles from random rotation matrixes
rand_aa = rm2aa(random_rots)
# axis_angles --> quaternions --> rotation_matrixes
rm_from_aa_through_quat = quat2rm(aa2quat(rand_aa))
# axis_angles --> rotation_matrixes
rm_from_aa = aa2rm(rand_aa)
rot_mat_reconstruction_diff = (rm_from_aa_through_quat - rm_from_aa).max(1)[0].max(1)[0]
print(f"angle_axis --> quaternions --> rotation_matrix vs angle_axis --> rotation_matrix\n"
      f"errors: {rot_mat_reconstruction_diff}")  # Small values ~1e-8

# Make sure that difference in reconstructed rotation matrixes actually produces different transforms
# When applied on randomly generated points
point_nb = 10
# Generate a batch of 10 random points per batch in 3D
rand_points = torch.rand(batch_size, 10, 3)

# Rotate points
points_rotated = rm_from_aa.bmm(rand_points.transpose(1, 2)).transpose(1, 2)
points_rotated_through_quat = rm_from_aa_through_quat.bmm(rand_points.transpose(1, 2)).transpose(1, 2)
reconstructed_distances = (points_rotated - points_rotated_through_quat).norm(2, -1).mean()
max_reconstructed_dists = (points_rotated - points_rotated_through_quat).norm(2, -1).max(1)[0]
print("Distances between points rotated using the two reconstructed rotations")
print(max_reconstructed_dists)  # errors in the range 1-2

I am puzzled because I compared the results from your angle_axis_to_rotation_matrix and a function I have been using for a long time which performs the same purpose, and they produce the same output, which makes me doubt whether there is actually an error despite the above-mentioned behavior.

[qiyan98]

Hi guys, sorry to bother here. It seems that this bug is still not fixed after one year. For anyone who is interested in the same functionality, try pytorch3d! In my experiments, kornia.angle_axis_to_rotation_matrix works fine and outputs the same results as cv2.Rodrigues(). However, kornia.rotation_matrix_to_angle_axis is definitely problematic and the cycle consistency is not guaranteed!

To bypass this, try pytorch3d.transforms.matrix_to_quaternion and pytorch3d.transforms.quaternion_to_axis_angle. The logic is the same as the kornia function kornia.angle_axis_to_rotation_matrix. matrix —> quaternion —> angle_axis.

[edgarriba]

@qiyan98 this will fixed after https://github.com/kornia/kornia/issues/483 😃