Using http snapshot image

I use Frigate and I have the detect role there using a low-res substream of my doorbell camera. It works great for person detection.

I currently use some simple automations along with Frigate to grab a higher res snapshot of my doorbell camera when a person is detected in a zone near the doorbell. Then through another automation in Home Assistant, I run Facebox on that image.

I’m considering trying Double Take to get better detection through Facebox and CompreFace, but I’m concerned that the image quality of anything coming from Frigate will be too poor for good detection.

I don’t want to switch the detect role on Frigate to the higher resolution as it just unnecessarily just burns CPU to detect people. So could I potentially use the http snapshot image option with Double Take to pull the higher res snapshots off of the doorbell camera and process those?

If so, what would you suggest as the best way of going about setting that up with HA automations and Double Take?

Thanks!

[jakowenko]

I’ll work on a beta build to support your use case and let you know when it’s ready. I’ll probably open up a new issue to track it.

There is a way to disable the Frigate MQTT processing and the latest processing. The latest is global to all Frigate cameras though. Setting latest to 0 will disable it and then setting mqtt to false will disable the snapshot topic processing.

frigate:
  attempts:
    # number of times double take will request a frigate latest.jpg for facial recognition
    latest: 0
    # number of times double take will request a frigate snapshot.jpg for facial recognition
    snapshot: 10
    # process frigate images from frigate/+/person/snapshot topics
    mqtt: false

[jakowenko]

The processing of the Frigate images / MQTT image is a little confusing and the MQTT one is done separately then the event driven one.

Frigate publishes a message to frigate/events

Double Take then begins to process this event using images from Frigate’s API. These images are /api/${camera}/latest.jpg and /api/events/${id}/snapshot.jpg. The height variable is passed to both of these images with the default being 500. Frigate allows you to request a cropped snapshot image as well. So if there are no masks configured in Double Take, then a cropped image for snapshot.jpg is passed to the detectors, rather than the full size one, which allows it to be processed quicker.

The construction of the Frigate URLs happens here and asking Frigate for a cropped snapshot image happens here.

Setting the height for the image like the below example will only apply to the latest.jpg and snapshot.jpg image when processed from a message on frigate/events

# frigate settings (default: shown below)
frigate:
  image:
    # height of frigate image passed for facial recognition
    height: 500

Frigate publishes a message to frigate/+/person/snapshot

Frigate publishes a JPEG encoded image to this topic when certain conditions are met on the Frigate side. There is no context around this image other than the camera name and JPEG frame. Double Take uses this image to process it through a single recognition loop. It’s only processed one time unless Frigate publishes a new image to that topic. Double Take doesn’t modify this image at all during the processing.

---

The context of this snapshot URL below and if configured like this is completely different. All this does is allow you to invoke this image via the Double Take API /api/camera/<camera-name>. No cropping/resizing is done to images processed this way.

cameras:
  doorbellcam:
    snapshot:
      url: http://192.168.1.10/onvif/snapshot

---

Hope this all makes sense. I can probably clarify it a bit better in the docs since it’s a bit confusing. With all of that said though, does it appear to be working as I described?