question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[Support]: No object detection at night with infrared

See original GitHub issue

Describe the problem you are having

Hi there,

I have realized that I don’t get any object detected in my outdoor cameras at night. I’m tracking several cats that roam my property and I have automated the sprinkler system to start for 1 min in the specific zone that the cat is seen. During the day, detection works flawlessly, but at night, cats are never detected (I even tried to go out myself and I was not detected as a person). I have a number of Dahua cams surrounding the property and the one with color-mode enabled seems to always catch the cat, while the infrared ones always miss it. For example, this is a snapshot of a missed cat:

image

How can I debug these types of events? I configured MQTT Explorer and left it running, however no MQTT events were found under that camera at that time. I just tried to lower the min_score for cats and dogs to 60 today, but my wife is recognized as a dog when wearing a ponytail 😃 Ideally, I would like to know what Frigate is doing when the cats are not recognized so I can tweak the config properly. Your help is very appreciated.

Thanks in advance!

Version

0.10.1-83481AF

Frigate config file

############################### MQTT ###############################
mqtt:
  host: REDACTED
  topic_prefix: frigate
  client_id: frigate
  user: REDACTED
  password: REDACTED
############################### GLOBAL CONFIG ###############################
rtmp:
  enabled: False
record:
  retain:
    days: 0
    mode: all
  events:
    pre_capture: 5
    post_capture: 10
    retain:
      default: 7
      mode: active_objects
      objects:
        person: 10
snapshots:
  enabled: True
  timestamp: True
  bounding_box: True
############################### INDIVIDUAL CAMERAS ###############################
cameras:
############################### FRONT RIGHT ###############################
  front_right:
    ffmpeg:
      inputs:
        - path: rtsp://REDACTED/cam/realmonitor?channel=1&subtype=0
          roles:
            - detect
            - record
      output_args:
        # Optional: output args for detect streams (default: shown below)
        #detect: -f rawvideo -pix_fmt yuv420p
        # Optional: output args for record streams (default: shown below))
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -c:a aac
    objects:
      track:
        - person
        - cat
        - dog
      filters:
        person:
          min_score: 0.75
          min_area: 5000
        cat:
          min_score: 0.60
        dog:
          min_score: 0.60
    detect:
      width: 2688
      height: 1520
      fps: 5
    zones:
      front_right_near:
        coordinates: 0,660,371,425,1367,589,2688,860,2688,1520,0,1520
        filters:
          person:
            min_area: 15000
    record:
      enabled: True
      events:
        required_zones: [front_right_near]
    snapshots:
      required_zones: [front_right_near]
############################### SIDE RIGHT ###############################
  side_right:
    ffmpeg:
      inputs:
        - path: rtsp://REDACTED/cam/realmonitor?channel=1&subtype=0
          roles:
            - detect
            - record
      output_args:
        # Optional: output args for detect streams (default: shown below)
        #detect: -f rawvideo -pix_fmt yuv420p
        # Optional: output args for record streams (default: shown below))
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -c:a aac
    objects:
      track:
        - person
        - cat
        - dog
      filters:
        person:
          min_score: 0.75
        cat:
          min_score: 0.60
        dog:
          min_score: 0.60
    detect:
      width: 2688
      height: 1520
      fps: 5
    zones:
      side_right_near:
        coordinates: 0,0,788,339,1401,265,2353,1152,2688,1006,2688,1520,0,1520
          #filters:
            #person:
              #min_area: 15000
    record:
      enabled: True
      events:
        required_zones: [side_right_near]
    snapshots:
      required_zones: [side_right_near]
############################### SIDE LEFT ###############################
  side_left:
    ffmpeg:
      inputs:
        - path: rtsp://REDACTED/cam/realmonitor?channel=1&subtype=0
          roles:
            - detect
            - record
      output_args:
        # Optional: output args for detect streams (default: shown below)
        #detect: -f rawvideo -pix_fmt yuv420p
        # Optional: output args for record streams (default: shown below))
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -c:a aac
    objects:
      track:
        - person
        - cat
        - dog
      filters:
        person:
          min_score: 0.75
        cat:
          min_score: 0.60
        dog:
          min_score: 0.60
    detect:
      width: 2688
      height: 1520
      fps: 5
    zones:
      #side_left_far: 
        #coordinates: 441,393,1099,423,2014,527,2688,676,2688,0,0,0,0,651
        side_left_near:
          coordinates: 2688,1520,2688,0,2544,0,2459,324,2062,147,1089,621,612,196,292,357,0,552,0,1520
          filters:
            person:
              min_area: 15000
    record:
      enabled: True
      events:
        required_zones: [side_left_near]
    snapshots:
      required_zones: [side_left_near]
############################### SIDE BACK RIGHT ###############################
  side_back_right:
    ffmpeg:
      inputs:
        - path: rtsp://REDACTED/cam/realmonitor?channel=1&subtype=1
          roles:
            - detect
        - path: rtsp://REDACTED/cam/realmonitor?channel=1&subtype=0
          roles:
            - record
      output_args:
        # Optional: output args for detect streams (default: shown below)
        #detect: -f rawvideo -pix_fmt yuv420p
        # Optional: output args for record streams (default: shown below))
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -c:a aac
    objects:
      track:
        - cat
        - dog
      filters:
        cat:
          min_score: 0.60
        dog:
          min_score: 0.60
    detect:
      width: 704
      height: 480
      fps: 5
    record:
      enabled: True
############################### BACK RIGHT ###############################
  back_right:
    ffmpeg:
      inputs:
        - path: rtsp://REDACTED/cam/realmonitor?channel=1&subtype=0
          roles:
            - detect
        - path: rtsp://REDACTED/cam/realmonitor?channel=1&subtype=0
          roles:
            - record
      output_args:
        # Optional: output args for detect streams (default: shown below)
        #detect: -f rawvideo -pix_fmt yuv420p
        # Optional: output args for record streams (default: shown below))
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -c:a aac
    objects:
      track:
        - cat
        - dog
      filters:
        cat:
          min_score: 0.60
        dog:
          min_score: 0.60
    detect:
      width: 2688
      height: 1520
      fps: 5
    motion:
      mask:
        - 2688,160,2688,825,2547,994,2382,750,2542,472
        - 2503,1176,2688,1001,2688,875,2469,1113
    record:
      enabled: True
############################### SIDE BACK LEFT ###############################
  side_back_left:
    ffmpeg:
      inputs:
        - path: rtsp://REDACTED/cam/realmonitor?channel=1&subtype=1
          roles:
            - detect
        - path: rtsp://REDACTED/cam/realmonitor?channel=1&subtype=0
          roles:
            - record
      output_args:
        # Optional: output args for detect streams (default: shown below)
        #detect: -f rawvideo -pix_fmt yuv420p
        # Optional: output args for record streams (default: shown below))
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -c:a aac
    objects:
      track:
        - cat
        - dog
      filters:
        cat:
          min_score: 0.60
        dog:
          min_score: 0.60
    detect:
      width: 704
      height: 480
      fps: 5
    record:
      enabled: True
############################### BACK LEFT ###############################
  back_left:
    ffmpeg:
      inputs:
        - path: rtsp://REDACTED/cam/realmonitor?channel=1&subtype=1
          roles:
            - detect
        - path: rtsp://REDACTED/cam/realmonitor?channel=1&subtype=0
          roles:
            - record
      output_args:
        # Optional: output args for detect streams (default: shown below)
        #detect: -f rawvideo -pix_fmt yuv420p
        # Optional: output args for record streams (default: shown below))
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -c:a aac
    objects:
      track:
        - cat
        - dog
      filters:
        cat:
          min_score: 0.60
        dog:
          min_score: 0.60
    detect:
      width: 704
      height: 480
      fps: 5
    record:
      enabled: True
############################### SIDE BACK RIGHT PEOPLE ###############################
  side_back_right_people:
    ffmpeg:
      inputs:
        - path: rtsp://REDACTED/cam/realmonitor?channel=1&subtype=1
          roles:
            - detect
        - path: rtsp://REDACTED/cam/realmonitor?channel=1&subtype=0
          roles:
            - record
      output_args:
        # Optional: output args for detect streams (default: shown below)
        #detect: -f rawvideo -pix_fmt yuv420p
        # Optional: output args for record streams (default: shown below))
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -c:a aac
    objects:
      track:
        - person
      filters:
        person:
          min_score: 0.75
    detect:
      enabled: False
      width: 704
      height: 480
      fps: 5
    record:
      enabled: False
    snapshots:
      enabled: False
  ############################### BACK RIGHT PEOPLE ###############################
  back_right_people:
    ffmpeg:
      inputs:
        - path: rtsp://REDACTED/cam/realmonitor?channel=1&subtype=0
          roles:
            - detect
        - path: rtsp://REDACTED/cam/realmonitor?channel=1&subtype=0
          roles:
            - record
      output_args:
        # Optional: output args for detect streams (default: shown below)
        #detect: -f rawvideo -pix_fmt yuv420p
        # Optional: output args for record streams (default: shown below))
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -c:a aac
    objects:
      track:
        - person
      filters:
        person:
          min_score: 0.75    
    detect:
      enabled: False
      width: 2688
      height: 1520
      fps: 5
    motion:
      mask:
        - 2688,160,2688,825,2547,994,2382,750,2542,472
        - 2503,1176,2688,1001,2688,875,2469,1113
    record:
      enabled: False
    snapshots:
      enabled: False
  ############################### SIDE BACK LEFT PEOPLE ###############################
  side_back_left_people:
    ffmpeg:
      inputs:
        - path: rtsp://REDACTED/cam/realmonitor?channel=1&subtype=1
          roles:
            - detect
        - path: rtsp://REDACTED/cam/realmonitor?channel=1&subtype=0
          roles:
            - record
      output_args:
        # Optional: output args for detect streams (default: shown below)
        #detect: -f rawvideo -pix_fmt yuv420p
        # Optional: output args for record streams (default: shown below))
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -c:a aac
    objects:
      track:
        - person
      filters:
        person:
          min_score: 0.75
    detect:
      enabled: False
      width: 704
      height: 480
      fps: 5
    record:
      enabled: False
    snapshots:
      enabled: False
  ############################### BACK LEFT PEOPLE ###############################
  back_left_people:
    ffmpeg:
      inputs:
        - path: rtsp://REDACTED/cam/realmonitor?channel=1&subtype=1
          roles:
            - detect
        - path: rtsp://REDACTED/cam/realmonitor?channel=1&subtype=0
          roles:
            - record
      output_args:
        # Optional: output args for detect streams (default: shown below)
        #detect: -f rawvideo -pix_fmt yuv420p
        # Optional: output args for record streams (default: shown below))
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -c:a aac
    objects:
      track:
        - person
      filters:
        person:
          min_score: 0.75
    detect:
      enabled: False
      width: 704
      height: 480
      fps: 5
    record:
      enabled: False
    snapshots:
      enabled: False
############################### DOORBELL ###############################
  doorbell:
    ffmpeg:
      inputs:
        - path: rtsp://REDACTED/cam/realmonitor?channel=1&subtype=0
          roles:
            - detect
            - record
      output_args:
        # Optional: output args for detect streams (default: shown below)
        #detect: -f rawvideo -pix_fmt yuv420p
        # Optional: output args for record streams (default: shown below)
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -c:a aac
    objects:
      track:
        - person
        - cat
        - dog
      filters:
        person:
          min_score: 0.75
        cat:
          min_score: 0.60
        dog:
          min_score: 0.60
    detect:
      width: 2560
      height: 1920
      fps: 5
    zones:
      doorbell_near:
        coordinates: 0,1920,2560,1920,2560,0,2006,0,2191,721,2198,1241,936,1439,923,976,919,0,0,0
        filters:
          person:
            min_area: 20000
    record:
      enabled: True
      events:
        required_zones: [doorbell_near]
    snapshots:
      required_zones: [doorbell_near]
############################### GARAGE ###############################
  garage:
    ffmpeg:
      inputs:
        - path: rtsp://REDACTED/cam/realmonitor?channel=1&subtype=1
          roles:
            - detect
      output_args:
        # Optional: output args for detect streams (default: shown below)
        #detect: -f rawvideo -pix_fmt yuv420p
        # Optional: output args for record streams (default: shown below)
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -c:a aac
    objects:
      track:
        - person
        - cat
        - dog
      filters:
        person:
          min_score: 0.75
    detect:
      width: 704
      height: 480
      fps: 3
    snapshots:
      enabled: False
############################### CPU CONFIG ###############################
detectors:
  coral:
    type: edgetpu
    device: usb

Relevant log output

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[2022-03-24 09:23:48] frigate.app                    INFO    : Starting Frigate (0.10.1-83481af)
[2022-03-24 09:23:48] frigate.app                    INFO    : Creating directory: /tmp/cache
Starting migrations
[2022-03-24 09:23:48] peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
[2022-03-24 09:23:48] peewee_migrate                 INFO    : There is nothing to migrate
[2022-03-24 09:23:48] frigate.app                    INFO    : Output process started: 222
[2022-03-24 09:23:48] ws4py                          INFO    : Using epoll
[2022-03-24 09:23:48] frigate.app                    INFO    : Camera processor started for front_right: 227
[2022-03-24 09:23:48] frigate.app                    INFO    : Camera processor started for side_right: 229
[2022-03-24 09:23:48] frigate.app                    INFO    : Camera processor started for side_left: 231
[2022-03-24 09:23:48] frigate.app                    INFO    : Camera processor started for side_back_right: 232
[2022-03-24 09:23:48] frigate.app                    INFO    : Camera processor started for back_right: 234
[2022-03-24 09:23:48] frigate.app                    INFO    : Camera processor started for side_back_left: 235
[2022-03-24 09:23:48] frigate.app                    INFO    : Camera processor started for back_left: 236
[2022-03-24 09:23:48] frigate.app                    INFO    : Camera processor started for side_back_right_people: 238
[2022-03-24 09:23:48] frigate.app                    INFO    : Camera processor started for back_right_people: 242
[2022-03-24 09:23:48] frigate.app                    INFO    : Camera processor started for side_back_left_people: 244
[2022-03-24 09:23:48] frigate.app                    INFO    : Camera processor started for back_left_people: 245
[2022-03-24 09:23:48] frigate.app                    INFO    : Camera processor started for doorbell: 246
[2022-03-24 09:23:48] frigate.app                    INFO    : Camera processor started for garage: 247
[2022-03-24 09:23:48] frigate.app                    INFO    : Capture process started for front_right: 248
[2022-03-24 09:23:48] frigate.app                    INFO    : Capture process started for side_right: 250
[2022-03-24 09:23:48] frigate.app                    INFO    : Capture process started for side_left: 253
[2022-03-24 09:23:48] frigate.app                    INFO    : Capture process started for side_back_right: 255
[2022-03-24 09:23:48] frigate.app                    INFO    : Capture process started for back_right: 258
[2022-03-24 09:23:48] frigate.app                    INFO    : Capture process started for side_back_left: 261
[2022-03-24 09:23:48] frigate.app                    INFO    : Capture process started for back_left: 265
[2022-03-24 09:23:48] frigate.app                    INFO    : Capture process started for side_back_right_people: 270
[2022-03-24 09:23:48] frigate.app                    INFO    : Capture process started for back_right_people: 275
[2022-03-24 09:23:48] frigate.app                    INFO    : Capture process started for side_back_left_people: 284
[2022-03-24 09:23:48] frigate.app                    INFO    : Capture process started for back_left_people: 302
[2022-03-24 09:23:48] frigate.app                    INFO    : Capture process started for doorbell: 306
[2022-03-24 09:23:48] frigate.app                    INFO    : Capture process started for garage: 318
[2022-03-24 09:23:48] ws4py                          INFO    : Using epoll
[2022-03-24 09:23:48] detector.coral                 INFO    : Starting detection process: 221
[2022-03-24 09:23:48] frigate.edgetpu                INFO    : Attempting to load TPU as usb
[2022-03-24 09:23:51] frigate.edgetpu                INFO    : TPU found

FFprobe output from your camera

Will try to provide later if needed

Frigate stats

{"back_left":{"camera_fps":5.1,"capture_pid":265,"detection_fps":0.6,"pid":236,"process_fps":5.0,"skipped_fps":0.0},"back_left_people":{"camera_fps":5.1,"capture_pid":302,"detection_fps":0.0,"pid":245,"process_fps":5.1,"skipped_fps":0.0},"back_right":{"camera_fps":5.2,"capture_pid":258,"detection_fps":0.0,"pid":234,"process_fps":5.2,"skipped_fps":0.0},"back_right_people":{"camera_fps":5.1,"capture_pid":275,"detection_fps":0.0,"pid":242,"process_fps":5.1,"skipped_fps":0.0},"detection_fps":9.7,"detectors":{"coral":{"detection_start":0.0,"inference_speed":14.13,"pid":221}},"doorbell":{"camera_fps":5.0,"capture_pid":306,"detection_fps":0.0,"pid":246,"process_fps":5.0,"skipped_fps":0.0},"front_right":{"camera_fps":5.0,"capture_pid":248,"detection_fps":3.2,"pid":227,"process_fps":5.0,"skipped_fps":0.0},"garage":{"camera_fps":3.1,"capture_pid":318,"detection_fps":0.0,"pid":247,"process_fps":3.1,"skipped_fps":0.0},"service":{"storage":{"/dev/shm":{"free":4125.4,"mount_type":"tmpfs","total":4171.4,"used":46.0},"/media/frigate/clips":{"free":183853.7,"mount_type":"ext4","total":240937.5,"used":44770.3},"/media/frigate/recordings":{"free":183853.7,"mount_type":"ext4","total":240937.5,"used":44770.3},"/tmp/cache":{"free":3804.4,"mount_type":"tmpfs","total":4171.4,"used":367.0}},"temperatures":{},"uptime":218,"version":"0.10.1-83481af"},"side_back_left":{"camera_fps":5.0,"capture_pid":261,"detection_fps":5.9,"pid":235,"process_fps":5.0,"skipped_fps":0.0},"side_back_left_people":{"camera_fps":5.1,"capture_pid":284,"detection_fps":0.0,"pid":244,"process_fps":5.1,"skipped_fps":0.0},"side_back_right":{"camera_fps":5.1,"capture_pid":255,"detection_fps":0.0,"pid":232,"process_fps":5.1,"skipped_fps":0.0},"side_back_right_people":{"camera_fps":5.1,"capture_pid":270,"detection_fps":0.0,"pid":238,"process_fps":5.1,"skipped_fps":0.0},"side_left":{"camera_fps":5.1,"capture_pid":253,"detection_fps":0.0,"pid":231,"process_fps":5.1,"skipped_fps":0.0},"side_right":{"camera_fps":5.1,"capture_pid":250,"detection_fps":0.0,"pid":229,"process_fps":5.1,"skipped_fps":0.0}}

Operating system

Debian

Install method

HassOS Addon

Coral version

USB

Network connection

Wired

Camera make and model

Dahua HDW2431TMN and Dahua T5442T-ZE

Any other information that may be helpful

No response

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:5

github_iconTop GitHub Comments

1reaction
NickM-27commented, Mar 24, 2022

It would be cool if Frigate could show us detection and motion boxes on 3rd party clips to see exactly what’s happening (like uploading a new clip and seeing what it discovers).

You can adjust your config to consider an mp4 clip as a camera to get a close idea of what the camera saw (NOTE: all config variables ex: detect motion need to be set to accurately see what happened previously). https://docs.frigate.video/contributing#2-create-a-local-config-file-for-testing

0reactions
HairyHookcommented, Mar 24, 2022

I will close this issue, thanks for your help. I’m aware of some new tools now that can help me track missing motion events. Cheers!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Object detection from UAV thermal infrared images and videos ...
However, the TIR images and videos can be taken at day and night, while the optical images can only be collected in the...
Read more >
Small Object Detection in Infrared Images - MDPI
We propose a framework for small object detection in the infrared domain based on YOLOv5, where target objects can be detected without ......
Read more >
CNN-Based Person Detection Using Infrared Images for Night ...
These deep learning-based approaches can enhance the object detection performance in infrared images since the increased model complexity ...
Read more >
The Best Night Vision Cameras of 2022 - Security.org
Best Camera with Both Infrared and color Night Vision ... No, Person, packages, vehicles, animal detection with Arlo Smart Home ...
Read more >
(PDF) Moving Object Detection in Night Time: A Survey
limitations of visual and CCD camera at night time, many. studies have been conducted on methods that detect objects. with infrared based ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found