[Feature Contribution] (Add Azure Kinect Camera to PyRobot)

I’m gonna customize the PyRobot with Microsoft’s new Azure Kinect Camera out of following considerations:

• The RealSense D435 camera has a very poor depth info accuracy, which make robotics application that relies on accurate depth info nearly impossible on PyRobot.
• Azure Kinect Camera can provide more information for robotics applications (like built-in body tracking parts in its ROS driver).
• Azure Kinect Camera itself is small enough to be mounted on PyRobot. So it just need to slightly modify the 3D printed part for camera mounting.
• The camera cost ($359) is low, which still fulfill the low-cost requirment of locobot.

Do you think it’s a good feature to be added into PyRobot?

If it is, my expected workflow of this feature is:

1. Rewrite the locobot_install_all.sh file, add installation of Azure Kinect SDK and its ROS Driver, and dependencies for calibration.
2. Rewrite the PyRobot’s files about camera and vision-related applications.
3. Add camera’s URDF file.
4. Add modified 3D printed parts files.

Is there any other aspects of working that I fail to consider?

[msr-peng]

Hey Kalyan,

Just a update. I uploaded a video about Rviz when did camera calibration here. It seems all motors values would suddenly return 0 (default value) periodically. That’s why the camera calibration failed. I guessed the issue may result from DYNAMIXEL U2D2 or DYNAMIXEL Power Hub. I’ll replace these hardwares and figure out whether it will work.

Thanks.

[kalyanvasudev]

Hi, yes Azure Kinect camera would be a great feature to add to PyRobot and LoCobot.

I agree, Azure Kinect could be a potentially better and relatively inexpensive alternative to Realsense D435. However, I am a little worried about the form factor (size and weight) of the Azure Kinect. In the sense that it might be a lot of weight for the pan and tilt motors on the LoCoBot to take. Alternatively, if that is the case, we can always remove pan and tilt motors on the LoCoBot and develop a version with Azzure Kinect rigidly mounted on it.

Overall, It is a great feature to add and see on LoCoBot and PyRobot.

Regarding your workflow ( We will be happy to help you and assist you in each and every step of it 😃 )

To begin with, your work flow looks good to me.

1. Yes, if possible make a separate script for Azure Kinect and call it in locobot_intall_all.sh through special sensor type arguments.

2. Yes, you would build a derived class on the existing pyrobot.camera class or pyrobot.locobot class. You might also need to modify locobot configuration files too.

3. Yes, you would make a copy of the existing URDF and modify it for LoCobot with Kinect.

4. I will get back to you on where exactly to add the 3-D printed parts files.

Other things you might be missing include, (Again, I will be happy to help you and assist along the way)

1. moveit_config package for the new URDF.
2. Tests
3. Modifying the launch file locobot_control/launch/main.launch
4. Making sure the existing SLAM, Vision algorithms on the PyRobot work for the new LoCoBot with Kinect.

Looking forward to this integration! Please let me know if you have anymore questions, doubts, etc.