VideoDevice returned image frames can contain mostly empty data

Describe the bug

While investigating the image color accuracy issue described in #1688, I noticed that my network was starting to have performance issues the more video devices i had running. At the time i was testing a Raspberry Pi 4 8GB running arm64 raspberry os with a low resolution night vision camera. I realised that most of the traffic from the camera arriving at my browser, was the image (around 25kb) followed by megabytes of 0’s.

After digging in further, I identified the issue is being caused by the code that process the captured frames, using the buffer size instead of the size of the image within the buffer. In my earlier testing with a different combination of camera/pi, this was not obvious as the images were of a similar size to the buffers, but with this low resolution camera, the issue became stark due to how much larger the resulting frames were compared to what they should have been.

From what I can tell, the size of the buffer changes dependent on the device being used, so using a high ram pi with a low res camera results in 99% of the buffer being empty. I’ve not confirmed this is the case, but certainly appears to correlate with my testing.

In my fork, I’ve ‘fixed’ this issue using the MemoryExtensions.TrimEnd(ReadOnlySpan<byte>) method to get the actual length of the data within the buffer to be returned, but that method is not available in netcoreapp2.1, so wasn’t sure that would be an acceptable fix for PR back.

Steps to reproduce

This can be reproduced using the MJPEG sample project in the repo, though it does need a device with a high memory to image size ratio to be obvious.

Expected behavior

The returned frame contains minimal additional space beyond the image data, only accounting for if ArrayPool is in use to reduce allocations (as the rented byte[] will always be larger than required)

Actual behavior

In some circumstances, the returned frame is 99.9% empty data

Versions used

dotnet --info
.NET SDK (reflecting any global.json):
 Version:   6.0.100-rc.1.21458.32
 Commit:    d7c22323c4

Runtime Environment:
 OS Name:     raspbian
 OS Version:  10
 OS Platform: Linux
 RID:         linux-arm
 Base Path:   /home/pi/dotnet/sdk/6.0.100-rc.1.21458.32/

Host (useful for support):
  Version: 6.0.0-rc.1.21451.13
  Commit:  d7619cd4b1

.NET SDKs installed:
  6.0.100-rc.1.21458.32 [/home/pi/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.0-rc.1.21452.15 [/home/pi/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.0-rc.1.21451.13 [/home/pi/dotnet/shared/Microsoft.NETCore.App]

• Version of System.Device.Gpio: built from main
• Version of Iot.Device.Bindings built from main

[joperezr]

Sorry that we missed that @CodedBeard, we recently recycled some of our labels and added some issue management automation which is probably why we missed your update and the bot commented to notify there wasn’t enough info here.

@raffaeler or @Ellerbach does anyone of you have the device that @CodedBeard pointed out above (the NOIR night vision camera) so we can test if the issue repros for us as well?

[raffaeler]

I am closing this in favor of https://github.com/dotnet/iot/issues/1799 since it has the same root cause. If you feel that instead they don’t share the same cause, please feel free to reopen this.