Detection stops working with ZeroDivisionError exception

Describe the problem you are having

Hello,

Thanks for Frigate!

I have a weird issue on latest beta: some cameras (between 1 and 3 in general over 6) stop working after some time (few hours) with ZeroDivisionError exception. No relevant logs prior to this exception.

I through it was related to HW acceleration (as I enabled it recently) but the problem still happen with HW acceleration disabled.

Any hints?

Version

0.11.0-be7b858

Frigate config file

[...]
ffmpeg:
  hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p

cameras:
  Camera1:
    ffmpeg:
      inputs:
        - path: rtsp://XXX:XXX@IP:554/11
          roles:
            - detect
            - record
      output_args:
        # Enable audio
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -c:a aac
    detect:
      width: 1920
      height: 1080
      fps: 5
    record:
      enabled: true
      retain:
        days: 4
      events:
        pre_capture: 2
        post_capture: 2
        retain:
          default: 2
          objects:
            person: 7
            car: 7
    objects:
      track:
        - person
[...]

Relevant log output

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[2022-09-20 12:57:12] frigate.app                    INFO    : Starting Frigate (0.11.0-be7b858)
Starting migrations
[2022-09-20 12:57:13] peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
[2022-09-20 12:57:13] peewee_migrate                 INFO    : There is nothing to migrate
[2022-09-20 12:57:13] detector.coral                 INFO    : Starting detection process: 215
[2022-09-20 12:57:13] frigate.edgetpu                INFO    : Attempting to load TPU as pci
[2022-09-20 12:57:13] ws4py                          INFO    : Using epoll
[2022-09-20 12:57:13] frigate.app                    INFO    : Output process started: 217
[2022-09-20 12:57:13] frigate.app                    INFO    : Camera processor started for Camera1: 223
[2022-09-20 12:57:13] frigate.app                    INFO    : Camera processor started for Camera2: 225
[2022-09-20 12:57:13] frigate.app                    INFO    : Camera processor started for Camera3: 237
[2022-09-20 12:57:13] frigate.edgetpu                INFO    : TPU found
[2022-09-20 12:57:13] frigate.app                    INFO    : Camera processor started for Camera4: 239
[2022-09-20 12:57:13] frigate.app                    INFO    : Camera processor started for Camera5: 240
[2022-09-20 12:57:13] frigate.app                    INFO    : Camera processor started for Camera6: 242
[2022-09-20 12:57:13] frigate.app                    INFO    : Camera processor started for Camera7: 243
[2022-09-20 12:57:13] frigate.app                    INFO    : Capture process started for Camera1: 245
[2022-09-20 12:57:13] frigate.app                    INFO    : Capture process started for Camera2: 249
[2022-09-20 12:57:13] frigate.app                    INFO    : Capture process started for Camera3: 252
[2022-09-20 12:57:13] frigate.app                    INFO    : Capture process started for Camera4: 255
[2022-09-20 12:57:13] frigate.app                    INFO    : Capture process started for Camera5: 261
[2022-09-20 12:57:13] frigate.app                    INFO    : Capture process started for Camera6: 266
[2022-09-20 12:57:13] frigate.app                    INFO    : Capture process started for Camera7: 273
[2022-09-20 12:57:13] ws4py                          INFO    : Using epoll
Process camera_processor:Camera1:
Traceback (most recent call last):
  File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/frigate/frigate/video.py", line 388, in track_camera
    process_frames(
  File "/opt/frigate/frigate/video.py", line 628, in process_frames
    detect(
  File "/opt/frigate/frigate/video.py", line 465, in detect
    ratio = width / height
ZeroDivisionError: division by zero
Process camera_processor:Camera2:
Traceback (most recent call last):
  File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/frigate/frigate/video.py", line 388, in track_camera
    process_frames(
  File "/opt/frigate/frigate/video.py", line 628, in process_frames
    detect(
  File "/opt/frigate/frigate/video.py", line 465, in detect
    ratio = width / height
ZeroDivisionError: division by zero
Process camera_processor:Camera5:
Traceback (most recent call last):
  File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/frigate/frigate/video.py", line 388, in track_camera
    process_frames(
  File "/opt/frigate/frigate/video.py", line 628, in process_frames
    detect(
  File "/opt/frigate/frigate/video.py", line 465, in detect
    ratio = width / height
ZeroDivisionError: division by zero
[2022-09-20 18:09:58] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:36320]
[2022-09-20 18:11:49] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:44386]
[2022-09-20 18:11:54] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:44386]
[2022-09-20 18:11:57] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:55774]
[2022-09-20 18:15:30] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:55774]

FFprobe output from your camera

N/A

Frigate stats

No response

Operating system

Debian

Install method

Docker CLI

Coral version

M.2

Network connection

Wired

Camera make and model

Various

Any other information that may be helpful

• On Cameras page, image is freezed
• Live stream is not working
• Detection is not working
• Recording is working
• In stats, frozen cameras fps are still at 5fps
• LIBVA_DRIVER_NAME set to i965
• Frigate running in k3s v1.24.4+k3s1
• CPU is a J4125

Furthermore, is there any way to detect when the problem occurs to restart the pod? (with a liveness probe) Log pattern: no log, /api/stats: fps still > 0, any other ideas? EDIT: There is trivial log pattern (match ZeroDivisionError) but it’s not easily accessible using a liveness probe…

Thanks

[NickM-27]

@fl42 actually I missed something and the log won’t show on that build. Can you please try crzynik/frigate:zero-division-check-t and again post if you see any of those errors. Appreciate your help with tihs.

[NickM-27]

@fl42 Can you please try once more? We are still not 100% sure where this issue originated, can you please try crzynik/frigate:zero-division-check for some time and let me know if you see any of those error logs?