Jetson Nano Unknown decoder 'h264_nvmpi'

Describe the bug Basically I am running this on a jetson nano, I have compiled ffmpeg with the jocover/jetson-ffmpeg package. I have also confirmed that it is properly showing by running ffmpeg -encoders |grep 264 and I see it there. Frigate say that it is an unknown decoder.

Version of frigate 0.8.4-5043040

Config file Include your full config file wrapped in triple back ticks.

detectors:
  coral:
    type: edgetpu
    device: pci
ffmpeg:
    global_args: -hide_banner -loglevel warning
    hwaccel_args: []
    input_args: -c:v h264_nvmpi -vsync drop
    output_args:
        detect: -f rawvideo -pix_fmt rgb24
mqtt:
  host: 192.168.0.000
  user: **********
  password: *********
objects:
  # Optional: list of objects to track from labelmap.txt (default: shown below)
  track:
    - person
    - car
    - truck
    - dog
    - horse
cameras:
  front_left:
    ffmpeg:
      inputs:
        - path: rtsp://admin:$ecret123@192.168.0.000/Streaming/Channels/101
          roles:
            - detect
            - rtmp
            - clips
    width: 2688
    height: 1520
    fps: 1
    clips:
      enabled: true
    motion:
      mask:
        - 0,0,690,0,499,35,0,479
        - 2688,0,2688,277,2320,142,1623,0
        #Removing timestamp
        - 114,94,104,153,857,177,888,97
  front_right:
    ffmpeg:
      inputs:
        - path: rtsp://admin:$ecret123@192.168.0.000/Streaming/Channels/301
          roles:
            - detect
            - rtmp
            - clips
    width: 2688
    height: 1520
    fps: 1
    clips:
      enabled: true
    motion:
      mask:
        - 2688,0,2688,548,2504,316,2372,284,1810,156,1502,132,1179,118,482,135,146,180,0,243,0,0
        #Removing timestamp
        - 114,94,104,153,857,177,888,97
  back_left:
    ffmpeg:
      inputs:
        - path: rtsp://admin:$ecret123@192.168.0.000/Streaming/Channels/201
          roles:
            - detect
            - rtmp
    width: 2688
    height: 1520
    fps: 1
    motion:
      mask:
        - 191,496,83,55,836,69,1044,205,538,1304
  backyard:
    ffmpeg:
      inputs:
        - path: rtsp://192.168.0.000:554/live0
          roles:
            - detect
            - rtmp
    width: 1920
    height: 1080
    fps: 1
  garage:
    ffmpeg:
      inputs:
        - path: rtsp://192.168.0.000:554/live0
          roles:
            - detect
            - rtmp
    width: 1920
    height: 1080
    fps: 1

Frigate container logs

frigate.video                  INFO    : backyard: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video                  INFO    : backyard: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video                  INFO    : backyard: ffmpeg process is not running. exiting capture thread...
ffmpeg.garage.detect           ERROR   : Unknown decoder 'h264_nvmpi'

Frigate stats

{
  "back_left": {
    "camera_fps": 0.0, 
    "capture_pid": 63, 
    "detection_fps": 0.0, 
    "pid": 44, 
    "process_fps": 0.0, 
    "skipped_fps": 0.0
  }, 
  "backyard": {
    "camera_fps": 0.0, 
    "capture_pid": 66, 
    "detection_fps": 0.0, 
    "pid": 55, 
    "process_fps": 0.0, 
    "skipped_fps": 0.0
  }, 
  "detection_fps": 0.0, 
  "detectors": {
    "coral": {
      "detection_start": 0.0, 
      "inference_speed": 10.0, 
      "pid": 36
    }
  }, 
  "front_left": {
    "camera_fps": 0.0, 
    "capture_pid": 57, 
    "detection_fps": 0.0, 
    "pid": 39, 
    "process_fps": 0.0, 
    "skipped_fps": 0.0
  }, 
  "front_right": {
    "camera_fps": 0.0, 
    "capture_pid": 58, 
    "detection_fps": 0.0, 
    "pid": 41, 
    "process_fps": 0.0, 
    "skipped_fps": 0.0
  }, 
  "garage": {
    "camera_fps": 0.0, 
    "capture_pid": 70, 
    "detection_fps": 0.0, 
    "pid": 56, 
    "process_fps": 0.0, 
    "skipped_fps": 0.0
  }, 
  "service": {
    "storage": {
      "/dev/shm": {
        "free": 1047.9, 
        "mount_type": "tmpfs", 
        "total": 1048.6, 
        "used": 0.7
      }, 
      "/media/frigate/clips": {
        "free": 35464.0, 
        "mount_type": "ext4", 
        "total": 62716.1, 
        "used": 24542.0
      }, 
      "/media/frigate/recordings": {
        "free": 35464.0, 
        "mount_type": "ext4", 
        "total": 62716.1, 
        "used": 24542.0
      }, 
      "/tmp/cache": {
        "free": 35464.0, 
        "mount_type": "ext4", 
        "total": 62716.1, 
        "used": 24542.0
      }
    }, 
    "uptime": 572, 
    "version": "0.8.4-5043040"
  }
}

FFprobe from your camera

Run the following command and paste output below

I don't think this is useful as my streams show fine without hwaccell

Screenshots Video is all green with this option on

Computer Hardware

• OS: JetPack Ubuntu
• Install method: Docker Compose
• Virtualization: N/A
• Coral Version: m2 ekey
• Network Setup: Wired

Camera Info: Not relevant

[e1z0]

https://github.com/e1z0/jetson-frigate i’ve do some research on it also.

[0mdb]

I’m still picking away at this to see if this makes sense before submitting a pull request. This nvidia hardware tends to have a long life so I think once it’s figured out it shouldn’t be too bad to maintain. I’m still using my shield from 2016 for example.

I tried to make jetson-ffmpeg work with no luck, this likely to my limited experience building docker images–good opportunity to learn I suppose. I believe I found the compiled image mentioned earlier in this thread from the jetson-frigate project here: https://hub.docker.com/r/nulldevil/frigate/tags

I also noticed the same very limited GPU use as @KillahB33. More like flickering really. However I noticed zero NVDEC use which is really what we’re after. So based on that I wouldn’t write this off just yet.

If we are mostly concerned with NVDEC (hardware decoding) and can forgo hardware encoding a potentially simpler direction could be to use the Nvidia ffmpeg and just install it with apt.

If the correct way to do this would be to create a Dockerfile.ffmpeg.aarch64jetson. The Nvidia l4t base image could be used. You have to align the l4t image with the jetpack cuda version so for me it would be: R32.6.1 here: nvcr.io/nvidia/l4t-base:r32.6.1 I think you could also expose cuda that way but not sure if that’s valuable.

I did some limited testing with my camera streams and nvidia ffmpeg using that in the Nvidia l4t base image and it works great. There is also a version with apt sources enabled which may be easier to incorporate: https://hub.docker.com/r/mdegans/l4t-base

Next I think I’ll try to incorporate the Nvidia FFMPEG directly into the base like this:

Anyways, let me know I’m barking up the wrong tree.