[Support]: current_zones MQTT object sometimes empty

Describe the problem you are having

I’m logging the MQTT messages of the events, and sometimes the “current_zones” object is empty even when the object is in a zone that should detect that object.

Version

0.11.1-2eada21

Frigate config file

# yaml-language-server: $schema=http://10.0.1.62:5000/api/config/schema

#Example config here:
#https://docs.frigate.video/configuration/index/

mqtt:
  host: 10.0.1.62
  user: 
  password: 

detectors:
  coral:
    type: edgetpu
    device: usb

#https://docs.frigate.video/configuration/camera_specific
# Reference - Blue Iris RTSP Cameras
ffmpeg:
  # todo - see if removing some of this helps playback
  hwaccel_args:
    - -hwaccel
    - vaapi
    - -hwaccel_device
    - /dev/dri/renderD128
    - -hwaccel_output_format
    - yuv420p
  input_args: -avoid_negative_ts make_zero -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1

rtmp:
  enabled: True # <-- RTMP should be disabled if your stream is not H264

snapshots:
  enabled: True
  clean_copy: False
  crop: True
  required_zones:
    - sidewalk_zone
    - driveway_car_zone
    - driveway_person_zone
    - porch_zone
    - yard_zone
  bounding_box: True
  retain:
  # Required: Default retention days (default: shown below)
    default: 1

record:
  enabled: False
  events:
    objects:
      - person
      - car
      - dog
      - cat
    required_zones:
      - sidewalk_zone
      - driveway_car_zone
      - driveway_person_zone
      - porch_zone
      - yard_zone
      #- street_zone
    retain:
      default: 5
      mode: active_objects

objects:
  # Optional: list of objects to track from labelmap.txt (default: shown below)
  track:
      - person
  

detect:
  enabled: True
  width: 1280
  height: 720

cameras:
  baby_cam:
    ffmpeg:      
      inputs:
        - path: rtsp://10.0.1.35:3765/eufy_cam
        #- path: /config/www/debug/driveway_sub.mp4 # debug only
        #  input_args: -re -stream_loop -1 -fflags +genpts # debug only
          roles:
            - detect 
    detect:
      width: 1920
      height: 1080
      fps: 1
    objects:
      track:
        - person
    mqtt:
      enabled: False
      

  driveway_cam:
    ffmpeg:      
      inputs:
        - path: rtsp://10.0.1.35:3765/driveway_sub
        #- path: /config/www/debug/driveway_sub.mp4 # debug only
        #  input_args: -re -stream_loop -1 -fflags +genpts # debug only
          roles:
            - detect
            - rtmp
            - record    

    objects:
      track:
        - person
        - car
      # NOTE: This mask is COMBINED with the object type specific mask below
      mask:
        - 1280,720,1280,320,1249,381,1209,384,1190,421,1204,451,1200,525,1160,720
        - 1061,0,1280,0,1280,343,1160,283,1066,285,992,259,943,208,932,167,765,135,506,77,494,0
        - 20,207,45,207,44,188,20,188
      filters:
        car:
          mask: 
            - 820,215,983,248,1082,163,1280,322,1280,572,1027,414,920,419,655,297,546,198
          
    motion:
      mask:
        - 976,358,1160,411,1140,471,1167,502,1041,720,1280,720,1280,474,1280,0,0,0,0,27,180,28,491,56,518,111,517,193,563,220,641,233,709,263,779,270,834,293,899,327
    
    zones:
      # Required: name of the zone
      # NOTE: This must be different than any camera names, but can match with another zone on another
      #       camera.
      sidewalk_zone:
        # Required: List of x,y coordinates to define the polygon of the zone.
        # NOTE: Presence in a zone is evaluated only based on the bottom center of the objects bounding box.
        coordinates: 461,278,762,271,756,204,0,188,0,300
        # Optional: List of objects that can trigger this zone (default: all tracked objects)
        objects:
          - person
        # Optional: Zone level object filters.
        # NOTE: The global and camera filters are applied upstream.
        filters:
          person:
            min_area: 4225
            max_area: 29000
            min_score: 0.5
            threshold: 0.6
            # Optional: mask to prevent this object type from being detected in certain areas (default: no mask)
            # Checks based on the bottom center of the bounding box of the object
            #mask: 0,0,0,0,0,0
      driveway_car_zone:
        coordinates: 177,720,1016,720,1059,683,1042,662,1115,525,843,403,690,354,527,244,343,246,155,254
        objects:
          - car
        filters:
          car:
            min_area: 30000
            max_area: 500000
            min_score: 0.5
            threshold: 0.6
      street_zone:          
        coordinates: 418,282,559,278,527,195,951,226,941,179,782,140,483,96,195,80,0,84,0,189,144,181,154,295,257,290
        objects:
          - car
          - person
        filters:
          car:
            min_area: 1800
            max_area: 755000
            min_score: 0.5
            threshold: 0.6
          person:
            min_area: 600
            max_area: 15000
            min_score: 0.5
            threshold: 0.6
      driveway_person_zone:
        objects:
          - person
        coordinates: 0,720,1108,720,1200,525,1193,481,987,340,791,294,460,282,0,312
        filters:
          person:
            min_area: 7000
            max_area: 110000
            min_score: 0.5
            threshold: 0.6
      #no_alert_zone:
      #  coordinates: 1100,720,1170,540,1056,475,773,481,633,522,868,720

  front_yard_cam:
    ffmpeg:
      inputs:
        - path: rtsp://10.0.1.35:3765/front_yard_sub        
        #- path: /config/www/debug/front_yard_sub.mp4 # debug only
        #  input_args: -re -stream_loop -1 -fflags +genpts # debug only
          roles:
            - detect
            - rtmp
            - record

    objects:
      mask:
        - 1280,70,1280,0,1077,0,0,0,0,584,396,342,326,53,667,34,723,96
    motion:
      mask:
        - 1065,691,789,666,579,633,798,321,629,297,218,666,0,670,0,720,1136,720
        - 0,0,1280,0,1280,106,908,113,0,125

    zones:
      # Required: name of the zone
      # NOTE: This must be different than any camera names, but can match with another zone on another
      #       camera.
      yard_zone:
        # Required: List of x,y coordinates to define the polygon of the zone.
        # NOTE: Presence in a zone is evaluated only based on the bottom center of the objects bounding box.
        coordinates: 1280,720,1280,222,515,219,383,223,386,279,413,345,492,355,660,389,287,720
        # Optional: List of objects that can trigger this zone (default: all tracked objects)
        objects:
          - person
          - cat
          - dog
        # Optional: Zone level object filters.
        # NOTE: The global and camera filters are applied upstream.
        filters:
          person:
            min_area: 8000
            max_area: 120000
            min_score: 0.5
            threshold: 0.6
            # Optional: mask to prevent this object type from being detected in certain areas (default: no mask)
            # Checks based on the bottom center of the bounding box of the object
            #mask: 0,0,0,0,0,0
      sidewalk_zone:
        coordinates: 1280,212,941,212,577,212,613,155,843,142,1280,139
        objects:
          - person
        filters:
          person:
            min_area: 4200
            max_area: 19500
            min_score: 0.5
            threshold: 0.6
      porch_zone:
        coordinates: 285,720,609,410,404,352,0,557,0,720
        objects:
          - person
          - cat
          - dog
        # Optional: Zone level object filters.
        # NOTE: The global and camera filters are applied upstream.
        filters:
          person:
            min_area: 10000
            max_area: 120000
            min_score: 0.5
            threshold: 0.6
     # driveway_zone:
      #  coordinates: 316,203,360,354,527,342,654,310,726,265,775,205,531,195
        #objects: # should use the default tracked objects above
          #- car
     # street_zone:
      #  coordinates: 569,171,766,162,1280,156,1280,64,428,105,360,120,347,144,358,167,408,165
       # objects:
         # - car
  
  doorbell_cam:
    ffmpeg:
      inputs:
        - path: rtsp://10.0.1.35:3765/doorbell_sub        
        #- path: /config/www/debug/doorbell_sub.mp4 # debug only
        #  input_args: -re -stream_loop -1 -fflags +genpts # debug only
          roles:
            - detect
            - rtmp
            - record

    motion:
      mask:
        - 0,1,0,93,561,167,658,266,619,317,619,415,700,428,682,720,1280,720,1280,0,2,3

    # testing this recommended config for double take (https://github.com/jakowenko/double-take)
    mqtt:
      timestamp: False
      bounding_box: False
      crop: True
      quality: 100
      height: 500

    zones:
      # Required: name of the zone
      # NOTE: This must be different than any camera names, but can match with another zone on another
      #       camera.
      porch_zone:
        # Required: List of x,y coordinates to define the polygon of the zone.
        # NOTE: Presence in a zone is evaluated only based on the bottom center of the objects bounding box.
        coordinates: 0,720,983,720,1035,544,541,496,428,452,0,485
        # Optional: List of objects that can trigger this zone (default: all tracked objects)
        objects:
          - person
          - cat
          - dog
        # Optional: Zone level object filters.
        # NOTE: The global and camera filters are applied upstream.
        #filters:
        #  person:
        #    min_area: 8000
        #    max_area: 120000
        #    min_score: 0.5
        #    threshold: 0.6

    
#logger:
#  default: debug

Relevant log output

Looking at the "before" objects, the first and third events have empty "current_zone" objects. In the first event, the "current_zones" should include the "driveway_car_zone" zone, and in the third event the "current_zones" should include the "street_zone" zone.

{"before":{"id":"1670460861.454207-v45ue5","camera":"driveway_cam","frame_time":1670460861.454207,"snapshot_time":0,"label":"car","sub_label":null,"top_score":0,"false_positive":true, "start_time":1670460861.454207,"end_time":null,"score":0.57421875,"box":[435,369,1074,708],"area":216621,"ratio":1.8849557522123894,"region":[246,0,1278,1032],"stationary":false,"motionless_count":0,"position_changes":0,"current_zones":[],"entered_zones":[],"has_clip":false,"has_snapshot":false},"after":{"id":"1670460861.454207-v45ue5","camera":"driveway_cam","frame_time":1670460862.198257,"snapshot_time":1670460862.198257,"label":"car","sub_label":null,"top_score":0.74609375,"false_positive":false,"start_time":1670460861.454207,"end_time":null,"score":0.76171875,"box":[358,313,988,718],"area":255150,"ratio":1.5555555555555556,"region":[20,0,1260,1240],"stationary":false,"motionless_count":3,"position_changes":1,"current_zones":["driveway_car_zone"],"entered_zones":["driveway_car_zone"],"has_clip":false,"has_snapshot":true},"type":"new"}
{"before":{"id":"1670460861.454207-v45ue5","camera":"driveway_cam","frame_time":1670460862.198257,"snapshot_time":1670460862.198257,"label":"car","sub_label":null,"top_score":0.74609375,"false_positive":false,"start_time":1670460861.454207,"end_time":null,"score":0.76171875,"box":[358,313,988,718],"area":255150,"ratio":1.5555555555555556,"region":[20,0,1260,1240],"stationary":false,"motionless_count":3,"position_changes":1,"current_zones":["driveway_car_zone"],"entered_zones":["driveway_car_zone"],"has_clip":false,"has_snapshot":true}, "after":{"id":"1670460861.454207-v45ue5","camera":"driveway_cam","frame_time":1670460868.936844,"snapshot_time":1670460862.198257,"label":"car","sub_label":null,"top_score":0.74609375,"false_positive":false,"start_time":1670460861.454207,"end_time":null,"score":0.57421875,"box":[237,97,361,244], "area":18228,"ratio":0.8435374149659864,"region":[0,0,1392,1392],"stationary":false,"motionless_count":0,"position_changes":1,"current_zones":[],"entered_zones":["driveway_car_zone"],"has_clip":false,"has_snapshot":true},"type":"update"}
{"before":{"id":"1670460861.454207-v45ue5","camera":"driveway_cam","frame_time":1670460868.936844,"snapshot_time":1670460862.198257,"label":"car","sub_label":null,"top_score":0.74609375,"false_positive":false,"start_time":1670460861.454207,"end_time":null,"score":0.57421875,"box":[237,97,361,244],"area":18228, "ratio":0.8435374149659864,"region":[0,0,1392,1392],"stationary":false,"motionless_count":0,"position_changes":1,"current_zones":[],"entered_zones":["driveway_car_zone"],"has_clip":false,"has_snapshot":true}, "after":{"id":"1670460861.454207-v45ue5","camera":"driveway_cam","frame_time":1670460875.598356,"snapshot_time":1670460862.198257,"label":"car","sub_label":null,"top_score":0.74609375,"false_positive":false,"start_time":1670460861.454207,"end_time":null,"score":0.71484375,"box":[107,30,405,139], "area":32482,"ratio":2.7339449541284404,"region":[0,0,604,604],"stationary":false,"motionless_count":7,"position_changes":1,"current_zones":["street_zone"],"entered_zones":["driveway_car_zone","street_zone"],"has_clip":false,"has_snapshot":true},"type":"update"}
{"before":{"id":"1670460861.454207-v45ue5","camera":"driveway_cam","frame_time":1670460875.598356,"snapshot_time":1670460862.198257,"label":"car","sub_label":null,"top_score":0.74609375,"false_positive":false,"start_time":1670460861.454207,"end_time":null,"score":0.71484375,"box":[107,30,405,139],"area":32482, "ratio":2.7339449541284404,"region":[0,0,604,604],"stationary":false,"motionless_count":7,"position_changes":1,"current_zones":["street_zone"],"entered_zones":["driveway_car_zone","street_zone"],"has_clip":false,"has_snapshot":true}, "after":{"id":"1670460861.454207-v45ue5","camera":"driveway_cam","frame_time":1670460876.057111,"snapshot_time":1670460862.198257,"label":"car","sub_label":null,"top_score":0.74609375,"false_positive":false,"start_time":1670460861.454207,"end_time":1670460881.098627,"score":0.53515625,"box":[57,32,287,142],  "area":25300,"ratio":2.090909090909091,"region":[0,0,668,668],"stationary":false,"motionless_count":1,"position_changes":1,"current_zones":["street_zone"],"entered_zones":["driveway_car_zone","street_zone"],"has_clip":false,"has_snapshot":true},"type":"end"}

FFprobe output from your camera

Not sure how to do this

Frigate stats

{"baby_cam":{"camera_fps":1.0,"capture_pid":230,"detection_fps":0.0,"pid":219,"process_fps":1.0,"skipped_fps":0.0},"detection_fps":0.0,"detectors":{"coral":{"detection_start":0.0,"inference_speed":10.09,"pid":216}},"doorbell_cam":{"camera_fps":5.0,"capture_pid":241,"detection_fps":0.0,"pid":229,"process_fps":5.0,"skipped_fps":0.0},"driveway_cam":{"camera_fps":5.1,"capture_pid":231,"detection_fps":0.0,"pid":220,"process_fps":5.1,"skipped_fps":0.0},"front_yard_cam":{"camera_fps":4.9,"capture_pid":235,"detection_fps":0.0,"pid":223,"process_fps":4.9,"skipped_fps":0.0},"service":{"latest_version":"0.11.1","storage":{"/dev/shm":{"free":4102.4,"mount_type":"tmpfs","total":4113.0,"used":10.6},"/media/frigate/clips":{"free":210672.3,"mount_type":"ext4","total":245373.3,"used":24689.9},"/media/frigate/recordings":{"free":210672.3,"mount_type":"ext4","total":245373.3,"used":24689.9},"/tmp/cache":{"free":4113.0,"mount_type":"tmpfs","total":4113.0,"used":0.0}},"temperatures":{},"uptime":90431,"version":"0.11.1-2eada21"}}

Operating system

HassOS

Install method

HassOS Addon

Coral version

USB

Network connection

Wired

Camera make and model

Dahua 6L04351PAGC8C7A

Any other information that may be helpful

No response

[J-Pipe]

Thank you Nick. I’ll try to play with that tonight and see what results I get.

[NickM-27]

I think the current_zone (and probably entered_zones) should be populated in the first MQTT message of the event.

I mean it is populated in the first message, just after because before the object was a true positive, the object was a false positive.

When an object is a false positive frigate is not confident that the object is what the label says. This means that object can not be part of a zone, among other things. The zone not being included in the before of a new event is not because it just isn’t included, the object does not have any zones attached since a false positive doesn’t have presence in zones.

• Is not a significant update when false positive
• If it’s not a significant update, then the event update is not called
• Update callback is where the previous state is updated to current state, meaning the TrackedObjects current state remains that way until significant_update is true

This means the fix is not as simple as including information that exists but isn’t presented, it’ll need to make large changes to either fundamentally change the way frigate treats false positives & zones or have it keep that information just for mqtt in a separate list. It will also be a large breaking change for all existing automations that work off the idea that an object being in a zone means it is not a false positive.

To me, this is an example of https://xyproblem.info/ where the mqtt information is being treated as the problem but in actuality the problem is that frigate does not have enough time to be confident that the car is actually a car before it enters the driveway. https://docs.frigate.video/guides/false_positives also explains how this works.

I can try to fix it and submit a PR if you think it will get accepted.

As you can probably tell, I disagree with this. However, I’m always open to hearing other’s opinions / perspectives. Also, what is accepted is up to @blakeblackshear and not me 👍