[Feature request] Rectification of stereo image pairs.

🚀 Feature

To complete the stereo pipeline in Kornia, we need functionality for rectifying images captured by a stereo camera setup.

This will be somewhat similiar to the OpenCV function cv2.initUndistortRectifyMap, but I don’t think we would initialize the rectification maps first. Instead we would just have the signature be:

def rectify(image, camera_matrix, R))
     return ...

def rectify_stereo_pair(left_image_batch, right_image_batch, left_camera_matrix, right_camera_matrix, R_left, R_right):
     left_image_batch_rectified = rectify(left_image_batch, left_camera_matrix, R_left)
     right_image_batch_rectified  = rectify(right_image_batch, right_camera_matrix, R_right)
     return left_image_batch_rectified, right_image_batch_rectified

Optionally we would also return the rectified camera matrices.

Note that OpenCV will undistortion and rectification in one step. I am unsure if this should be part of this first feature, but might be the way to go.

The kornia.geometry.undistort_points might be helpful here.

Motivation

To complete the stereo pipeline in Kornia, we need rectification functionality. That will allow the full stereo pipeline: Bayer Image -> Color Image -> Undistort -> Rectify -> Disparity map -> Point Cloud to be supported by Kornia.

Pitch

See above. I need to find good material for a rectification algorithm.

Here’s a starting point:

https://www.cs.cmu.edu/~16385/s17/Slides/13.1_Stereo_Rectification.pdf

[ducha-aiki]

It is very needed indeed! I agree that the best way is to create such functions alone and integrate later.

[copaah]

Added first go a reimplementing the algorithm from the paper. I get approximately same results, but need data from a real stereo rig to verify.