Consider use of `live_flv` ffmpeg flag with cameras that have timestamp issues

I’m the owner of some Reolink RLC-410 cameras, and I used the config linked in the docs to get a stream that decodes successfully. However, I had an issue where clips would have delayed frames followed by sped up sections. I suspect that this is caused by an interaction of the following options:

-use_wallclock_as_timestamps in frigate, a CPU (rather than tpu) detector, and

<clock offset='utc'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='yes'/>
</clock>

in my VM configuration. When an object is detected, the CPU detector puts the CPU under heavy load. This in turn delays processing of frames from the RTMP input, which means that the real time clock drifts from the actual correct inter-frame timing. When played back, the clip appears to freeze and speed up intermittently depending on the load when it was recorded. While this is certainly exacerbated by the VM, it’s possible for it to happen on any system that’s under load or bandwidth constrained.

I researched and tested out the live_flv option for ffmpeg. It appears to be designed for exactly this use case - timestamp discontinuities in livestreamed FLV. I specified this configuration:

      input_args:
        - -fflags
        - nobuffer
        - -flags
        - low_delay
        - -strict
        - experimental
        - -rw_timeout
        - '5000000'
        - -f
        - live_flv

Which then evaluates to this:

      "ffmpeg_cmds": [
        {
          "cmd": "ffmpeg -hide_banner -loglevel warning -fflags nobuffer -flags low_delay -strict experimental -rw_timeout 5000000 -f live_flv -i rtmp://10.0.0.253/bcs/channel0_main.bcs?channel=0&stream=0&user=frigate&password=********* -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an /tmp/cache/tree_driveway-%Y%m%d%H%M%S.mp4 -c copy -f flv rtmp://127.0.0.1/live/tree_driveway",
          "roles": [
            "clips",
            "rtmp"
          ]
        },
        {
          "cmd": "ffmpeg -hide_banner -loglevel warning -fflags nobuffer -flags low_delay -strict experimental -rw_timeout 5000000 -f live_flv -i rtmp://10.0.0.253/bcs/channel0_sub.bcs?channel=0&stream=0&user=frigate&password=********* -r 7 -f rawvideo -pix_fmt yuv420p pipe:",
          "roles": [
            "detect"
          ]
        }

This completely fixed my issues with clips - they’re rock solid now, perfectly smooth motion. I’ve been running this configuration for a few months. I also tried introducing artificial CPU load and starving the ffmpeg process of resources, and it handled those just fine. I’d suggest updating the documentation to use it for reolink cameras, or even changing it to the default frigate behavior for rtmp streams if some users test it on other cameras and it works well.

[skulumani]

I have two RLC-520 cameras. The following options are what works for me. Using that posted above @charlesmunger caused only a green image to display

input_args:
  - -avoid_negative_ts
  - make_zero
  - -flags
  - nobuffer
  - -flags
  - low_delay
  - -strict
  - experimental
  - --flags
  - +genpts+discardcorrupt
  - -rw_timeout
  - '5000000'
  - -use_wallclock_as_timestamps
  - '1'
  - -f
  - live_flv

[charlesmunger]

I have investigated further and found the issue - for me, at least, the nobuffer and low_delay options were the source of stuttering clips. I believe the reason your clips don’t stutter, @flaektrem , is because your input_args override the first -fflags with the second one to specify +genpts+discardcorrupt. That’s the general behavior of ffmpeg flags, although I can’t prove that’s how -fflags works. I now have stutter-free clips with:

  roles:
    - detect
  input_args:
    - -fflags
    - nobuffer
    - -flags
    - low_delay
    - -strict
    - experimental
    - -rw_timeout
    - '5000000'
    - -f
    - live_flv

and

roles:
  - record
input_args: &record_reolink_args
  - -strict
  - experimental
  - -rw_timeout
  - '5000000'
  - -f
  - live_flv

So try this configuration, and you might see lower delay on your detect stream, and a simpler setup for your record stream.