Libcamera stack support (features, APIs and discussions)

It’s been a while that the Raspicam camera stack has gone legacy. This means means the V4L2 drivers are not in place by default in the most recent versions of the Raspbian Operating System. In order to make our VideoDevice binding work, the legacy support must be set using raspi-config on the Raspberry PI. The complete overview on the camera support can be found here.

The new stack is called libcamera and is turned on by default in the OS. More info about libcamera:

• https://www.raspberrypi.com/news/an-open-source-camera-stack-for-raspberry-pi-using-libcamera/
• https://libcamera.org/entries/2020-05-05.html
• https://datasheets.raspberrypi.com/camera/raspberry-pi-camera-guide.pdf
• https://libcamera.org/

The Raspberry foundation also published a new set of applications/tools replacing the legacy raspistill/raspivid. The source code can be found at the following link: https://github.com/raspberrypi/libcamera-apps/tree/main/apps

The libcamera documentation also talks about a compatibility layer, but it is unclear whether and how it can be activated to make applications using V4L2 work without turning on the legacy Raspicam model. The most interesting links on this subject are here:

• https://forums.raspberrypi.com/viewtopic.php?t=322076
• https://lwn.net/Articles/881164/

In any case, it looks like libcamera is the future and it may be worth to build a new binding to fully exploit the potentials of the modern cameras connected to the Raspberry PI and similar devices.

Unfortunately, according the official libcamera account, they still do not provide a native C language API (but only C++), meaning that it makes far more difficult to create the interoperability code to provide a .NET binding. This would force the need to ship the binding with a certain amount of native code wrapping the official C++ interface. In turns this implies the need to ship nuget packages with different assemblies for each CPU architecture.

As a reminder, C++ does not provide a standard binary contract. This translates in the impossibility to guarantee the binary interop across different C++ compilers and often across different versions of the same C++ compiler.

I created this thread for a few reasons:

• Telling the community why the current VideoDevice binding is not working (unless you manually turn on the legacy support)
• Getting fresh ideas on the potential implementation of the libcamera binding (no promises)
• Seeing who is interested in working/helping on this implementation
• Eventually discussing the API to expose in the binding

[kbingham]

https://bugs.libcamera.org/show_bug.cgi?id=133 added to highlight that this needs development.

[kbingham]

I don’t know how you’re measuring performances, or what you’re facing. But you should certainly avoid any format conversions in software, and let the RPi use it’s hardware ISP… But it seems we might have to fix the format configuration for that.