Enabling Save_Clips: true caused FFMPEG to fail

Describe the bug When I enabled save_clips:true, the ffmpeg process will keep failing with below output in the logs. But if I disabled save_clips:false, the process will run smoothly without issues.

porch: ffmpeg sent a broken frame. something is wrong.
porch: ffmpeg process is not running. exiting capture thread...
Creating ffmpeg process...

Version of frigate 0.7.0 running as Home Assistant Addon

Config file

web_port: 5000
detectors:
  coral:
    type: edgetpu
    device: 'pci:0'
save_clips:
  clips_dir: /media/clips
mqtt:
  host: *redacted*
  topic_prefix: frigate
  user: *redacted*
  password: *redacted*
ffmpeg:
  hwaccel_args:
    - '-hwaccel'
    - vaapi
    - '-hwaccel_device'
    - /dev/dri/renderD128
    - '-hwaccel_output_format'
    - yuv420p
cameras:
  porch:
    ffmpeg:
      input: >-
        rtmp://admin:password@192.168.1.34:1935/bcs/channel0_sub.bcs?token=sdasdasd&channel=0&stream=0&user=admin&password=password
      global_args:
        - '-hide_banner'
        - '-loglevel'
        - info
      input_args:
        - '-avoid_negative_ts'
        - make_zero
        - '-fflags'
        - nobuffer
        - '-flags'
        - low_delay
        - '-strict'
        - experimental
        - '-fflags'
        - +genpts+discardcorrupt
        - '-vsync'
        - drop
        - '-use_wallclock_as_timestamps'
        - '1'
    take_frame: 1
    save_clips:
      enabled: true
      pre_capture: 30
    snapshots:
      show_timestamp: true
      draw_zones: false
    objects:
      track:
        - person
      filters:
        person:
          min_area: 3000
          max_area: 100000
          min_score: 0.6
          threshold: 0.85

FFMPEG Output Logs when running Save_Clips: true

Fontconfig error: Cannot load default config file
ffprobe -v panic -show_error -show_streams -of json "rtmp://admin:password@192.168.1.34:1935/bcs/channel0_sub.bcs?token=sdasdasd&channel=0&stream=0&user=admin&password=password"
Starting detection process: 27
Attempting to load TPU as pci:0
TPU found
On connect called
Creating ffmpeg process...
ffmpeg -hide_banner -loglevel info -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -use_wallclock_as_timestamps 1 -i rtmp://admin:password@192.168.1.34:1935/bcs/channel0_sub.bcs?token=sdasdasd&channel=0&stream=0&user=admin&password=password -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an -map 0 /cache/porch-%Y%m%d%H%M%S.mp4 -f rawvideo -pix_fmt yuv420p pipe:
Input #0, flv, from 'rtmp://admin:password@192.168.1.34:1935/bcs/channel0_sub.bcs?token=sdasdasd&channel=0&stream=0&user=admin&password=password':
  Metadata:
    displayWidth    : 640
    displayHeight   : 480
  Duration: 00:00:00.00, start: 1603097844.510000, bitrate: N/A
    Stream #0:0: Data: none
    Stream #0:1: Audio: aac, 16000 Hz, mono, fltp
    Stream #0:2: Video: h264, yuv420p(progressive), 640x480, 49 tbr, 1k tbn
[segment @ 0x55eb3a61f640] Opening '/cache/porch-20201019165730.mp4' for writing
[mp4 @ 0x55eb3a637100] Could not find tag for codec none in stream #0, codec not currently supported in container
Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:2 -> #0:1 (copy)
  Stream #0:2 -> #1:0 (h264 (native) -> rawvideo (native))
    Last message repeated 1 times
porch: ffmpeg sent a broken frame. something is wrong.
porch: ffmpeg process is not running. exiting capture thread...
Camera_process started for porch: 84
Starting process for porch: 84
 * Serving Flask app "detect_objects" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
WARNING: Invalid RefPicListX[] entry!!! It is not included in DPB
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x56512df64140] moov atom not found
/cache/porch-20201019165934.mp4: Invalid data found when processing input
bad file: porch-20201019165934.mp4
Creating ffmpeg process...
ffmpeg -hide_banner -loglevel info -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -use_wallclock_as_timestamps 1 -i rtmp://admin:password@192.168.1.34:1935/bcs/channel0_sub.bcs?token=sdasdasd&channel=0&stream=0&user=admin&password=password -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an -map 0 /cache/porch-%Y%m%d%H%M%S.mp4 -f rawvideo -pix_fmt yuv420p pipe:
Input #0, flv, from 'rtmp://admin:password@192.168.1.34:1935/bcs/channel0_sub.bcs?token=sdasdasd&channel=0&stream=0&user=admin&password=password':
  Metadata:
    displayWidth    : 640
    displayHeight   : 480
  Duration: 00:00:00.00, start: 1603097976.558000, bitrate: N/A
    Stream #0:0: Data: none
    Stream #0:1: Audio: aac, 16000 Hz, mono, fltp
    Stream #0:2: Video: h264, yuv420p(progressive), 640x480, 49 tbr, 1k tbn
[segment @ 0x5555f50a8840] Opening '/cache/porch-20201019165942.mp4' for writing
[mp4 @ 0x5555f4fa89c0] Could not find tag for codec none in stream #0, codec not currently supported in container
Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:2 -> #0:1 (copy)
  Stream #0:2 -> #1:0 (h264 (native) -> rawvideo (native))
    Last message repeated 1 times
porch: ffmpeg sent a broken frame. something is wrong.
porch: ffmpeg process is not running. exiting capture thread...
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x561db210a140] moov atom not found
/cache/porch-20201019165942.mp4: Invalid data found when processing input
bad file: porch-20201019165942.mp4
Creating ffmpeg process...
ffmpeg -hide_banner -loglevel info -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -use_wallclock_as_timestamps 1 -i rtmp://admin:password@192.168.1.34:1935/bcs/channel0_sub.bcs?token=sdasdasd&channel=0&stream=0&user=admin&password=password -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an -map 0 /cache/porch-%Y%m%d%H%M%S.mp4 -f rawvideo -pix_fmt yuv420p pipe:
No frames received from porch in 5 seconds. Exiting ffmpeg...
Waiting for ffmpeg to exit gracefully...
Input #0, flv, from 'rtmp://admin:password@192.168.1.34:1935/bcs/channel0_sub.bcs?token=sdasdasd&channel=0&stream=0&user=admin&password=password':
  Metadata:
    displayWidth    : 640
    displayHeight   : 480
  Duration: 00:00:00.00, start: 1603097988.918000, bitrate: N/A
    Stream #0:0: Data: none
    Stream #0:1: Audio: aac, 16000 Hz, mono, fltp
    Stream #0:2: Video: h264, yuv420p(progressive), 640x480, 1k tbr, 1k tbn
[segment @ 0x5610fb634d40] Opening '/cache/porch-20201019165954.mp4' for writing
[mp4 @ 0x5610fb63f1c0] Could not find tag for codec none in stream #0, codec not currently supported in container
Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:2 -> #0:1 (copy)
  Stream #0:2 -> #1:0 (h264 (native) -> rawvideo (native))
    Last message repeated 1 times
Exiting normally, received signal 15.
porch: ffmpeg sent a broken frame. something is wrong.
Exception in thread porch:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/opt/frigate/frigate/video.py", line 180, in run
    capture_frames(self.ffmpeg_process, self.name, self.frame_shape, self.frame_manager, self.frame_queue, self.take_frame,
  File "/opt/frigate/frigate/video.py", line 130, in capture_frames
    frame_bytes = ffmpeg_process.stdout.read(frame_size)
ValueError: read of closed file
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556dd1465140] moov atom not found
/cache/porch-20201019165954.mp4: Invalid data found when processing input
bad file: porch-20201019165954.mp4
Creating ffmpeg process...
ffmpeg -hide_banner -loglevel info -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -use_wallclock_as_timestamps 1 -i rtmp://admin:password@192.168.1.34:1935/bcs/channel0_sub.bcs?token=sdasdasd&channel=0&stream=0&user=admin&password=password -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an -map 0 /cache/porch-%Y%m%d%H%M%S.mp4 -f rawvideo -pix_fmt yuv420p pipe:
Input #0, flv, from 'rtmp://admin:password@192.168.1.34:1935/bcs/channel0_sub.bcs?token=sdasdasd&channel=0&stream=0&user=admin&password=password':
  Metadata:
    displayWidth    : 640
    displayHeight   : 480
  Duration: 00:00:00.00, start: 1603098005.426000, bitrate: N/A
    Stream #0:0: Data: none
    Stream #0:1: Audio: aac, 16000 Hz, mono, fltp
    Stream #0:2: Video: h264, yuv420p(progressive), 640x480, 1k tbr, 1k tbn
[segment @ 0x5629dfc97b80] Opening '/cache/porch-20201019170011.mp4' for writing
[mp4 @ 0x5629dfdb7f80] Could not find tag for codec none in stream #0, codec not currently supported in container
Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:2 -> #0:1 (copy)
  Stream #0:2 -> #1:0 (h264 (native) -> rawvideo (native))
    Last message repeated 1 times
porch: ffmpeg sent a broken frame. something is wrong.
porch: ffmpeg process is not running. exiting capture thread...
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x563240d08140] moov atom not found
/cache/porch-20201019170011.mp4: Invalid data found when processing input
bad file: porch-20201019170011.mp4

FFMPEG Output Logs when running Save_Clips: false

Creating ffmpeg process...
ffmpeg -hide_banner -loglevel info -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -use_wallclock_as_timestamps 1 -i rtmp://admin:password@192.168.1.34:1935/bcs/channel0_sub.bcs?token=sdasdasd&channel=0&stream=0&user=admin&password=password -f rawvideo -pix_fmt yuv420p pipe:
ffprobe -v panic -show_error -show_streams -of json "rtmp://admin:password@192.168.1.35:1935/bcs/channel0_sub.bcs?token=sdasdasd&channel=0&stream=0&user=admin&password=password"
Input #0, flv, from 'rtmp://admin:password@192.168.1.34:1935/bcs/channel0_sub.bcs?token=sdasdasd&channel=0&stream=0&user=admin&password=password':
  Metadata:
    displayWidth    : 640
    displayHeight   : 480
  Duration: 00:00:00.00, start: 1603097064.427000, bitrate: N/A
    Stream #0:0: Data: none
    Stream #0:1: Audio: aac, 16000 Hz, mono, fltp
    Stream #0:2: Video: h264, yuv420p(progressive), 640x480, 1k tbr, 1k tbn
Stream mapping:
  Stream #0:2 -> #0:0 (h264 (native) -> rawvideo (native))
Press [q] to stop, [?] for help
WARNING: Invalid RefPicListX[] entry!!! It is not included in DPB
frame=    0 fps=0.0 q=0.0 size=       0kB time=-577014:32:22.77 bitrate=  -0.0kbits/s speed=N/A    
frame=    0 fps=0.0 q=0.0 size=       0kB time=-577014:32:22.77 bitrate=  -0.0kbits/s speed=N/A    
frame=    0 fps=0.0 q=0.0 size=       0kB time=-577014:32:22.77 bitrate=  -0.0kbits/s speed=N/A    
Output #0, rawvideo, to 'pipe:':
  Metadata:
    displayWidth    : 640
    displayHeight   : 480
    encoder         : Lavf58.45.100
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 640x480, q=2-31, 3686400 kb/s, 1k fps, 1k tbn, 1k tbc
    Metadata:
      encoder         : Lavc58.91.100 rawvideo
[rawvideo @ 0x5614a3451380] Encoder did not produce proper pts, making some up.

Frigate debug stats

{"detectors":{"coral":{"detection_start":0.0,"inference_speed":6.25,"pid":27}},"porch":{"camera_fps":0.0,"detection_fps":0.0,"ffmpeg_pid":2363,"frame_info":{"detect":0.0,"process":0.0,"read":0.0},"pid":84,"process_fps":0.0,"read_start":0.0,"skipped_fps":0.0}}

FFprobe from your camera

{'streams': [{'index': 0, 'codec_type': 'data', 'codec_tag_string': '[0][0][0][0]', 'codec_tag': '0x0000', 'r_frame_rate': '0/0', 'avg_frame_rate': '0/0', 'time_base': '1/1000', 'start_pts': 0, 'start_time': '0.000000', 'disposition': {'default': 0, 'dub': 0, 'original': 0, 'comment': 0, 'lyrics': 0, 'karaoke': 0, 'forced': 0, 'hearing_impaired': 0, 'visual_impaired': 0, 'clean_effects': 0, 'attached_pic': 0, 'timed_thumbnails': 0}}, {'index': 1, 'codec_name': 'aac', 'codec_long_name': 'unknown', 'profile': '1', 'codec_type': 'audio', 'codec_time_base': '1/16000', 'codec_tag_string': '[0][0][0][0]', 'codec_tag': '0x0000', 'sample_fmt': 'fltp', 'sample_rate': '16000', 'channels': 1, 'channel_layout': 'mono', 'bits_per_sample': 0, 'r_frame_rate': '0/0', 'avg_frame_rate': '0/0', 'time_base': '1/1000', 'start_pts': 139444276, 'start_time': '139444.276000', 'disposition': {'default': 0, 'dub': 0, 'original': 0, 'comment': 0, 'lyrics': 0, 'karaoke': 0, 'forced': 0, 'hearing_impaired': 0, 'visual_impaired': 0, 'clean_effects': 0, 'attached_pic': 0, 'timed_thumbnails': 0}}, {'index': 2, 'codec_name': 'h264', 'codec_long_name': 'unknown', 'profile': '77', 'codec_type': 'video', 'codec_time_base': '0/2', 'codec_tag_string': '[0][0][0][0]', 'codec_tag': '0x0000', 'width': 640, 'height': 480, 'coded_width': 640, 'coded_height': 480, 'closed_captions': 0, 'has_b_frames': 0, 'pix_fmt': 'yuv420p', 'level': 50, 'field_order': 'progressive', 'refs': 1, 'is_avc': 'true', 'nal_length_size': '4', 'r_frame_rate': '30/1', 'avg_frame_rate': '0/1', 'time_base': '1/1000', 'start_pts': 139444173, 'start_time': '139444.173000', 'bits_per_raw_sample': '8', 'disposition': {'default': 0, 'dub': 0, 'original': 0, 'comment': 0, 'lyrics': 0, 'karaoke': 0, 'forced': 0, 'hearing_impaired': 0, 'visual_impaired': 0, 'clean_effects': 0, 'attached_pic': 0, 'timed_thumbnails': 0}}]}

Computer Hardware

• OS: Ubuntu
• Virtualization: bare metal
• Coral Version: PCIe
• Network Setup: Wired

Camera Info:

• Manufacturer: Reolink
• Model: RLC-410
• Resolution: 640x480 sub stream
• FPS: 7

[bm55]

Nice work. I also have few Reolink cameras and -map 0:2 flag or removing it fixes the save_clips issue.

[rhatguy]

oh boy…now looking at the logs on this Pi, its undervolting left and right even with an official rpi power supply. I guess that seals this efforts fate. The RPI 3b- is to slow to handle large resolution on its own, and at least with a hard drive and a coral attached doesn’t seem to be able to keep the power up:(