question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

RGB Alignment with Depth

See original GitHub issue

Start with the why:

In cases where both color is required for neural inference to work properly -and- the neural data needs to be perfectly aligned with the depth data, alignment between the RGB and depth information needs to exist.

A canonical example is doing semantic segmentation of color defects and needing to know their physical location. In this case, both color-based neural inference is needed (and it is per-pixel, since the network is a semantic segmentor) and depth information is needed to be aligned per-pixel.

Move to the how:

The Myriad X already has the capability to perform the transform to provide alignment between cameras. What is needed is a system for doing the calibration to determine this transform matrix, including at the differing resolutions of the color camera and the grayscale cameras (which are the source of the depth map).

Move to the what:

Provide a system for DepthAI users to calibrate the RGB camera against the right grayscale camera and the mechanism to do this alignment.

And for units with onboard cameras, improve our calibration system to perform this RGB-right calibration in the factory for all future production.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:8

github_iconTop GitHub Comments

2reactions
Luxonis-Brandoncommented, May 25, 2021
1reaction
TheBricktopcommented, Nov 13, 2021

Thank You ☺️

Read more comments on GitHub >

github_iconTop Results From Across the Web

RGB Depth alignment - DepthAI documentation - Luxonis
This example shows usage of RGB depth alignment. Since OAK-D has a color and a pair of stereo cameras, you can align depth...
Read more >
Align Depth and Color Frames – Depth and RGB Registration
The process of aligning the depth and the RGB frame is called “registration” and it is very easy to do (and the algorithm's...
Read more >
Depth image and RGB image alignment - YouTube
The depth image is aligned with the RGB image captured by our RGB -D camera. SDK provided by Vzense supports the pixel alignment...
Read more >
Illustration of the RGB-D alignment: the depth image (top-left ...
Illustration of the RGB-D alignment: the depth image (top-left) is aligned with the RGB image (top-right) captured by the Kinect at the same...
Read more >
RGB alignment/registration to depth frame. - Structure Sensor
For every possible u, v, z value I store the calculated rgb depth. At centimeter precision level it takes 2GB of ram but...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found