How to interpret event json

[blacknell]

I'm trying to work out why my triggers are failing or are over sensitive.

My automation looks at a person who moves from the driveway to the porch. But at night I get a lot of false positives and so I want to use scores and sub labels to help me

Here's some json I reviewed. You will note it is the 'end' frame.

On the snapshot in Frigate I see a bounding box for both the person and the face so I know it knows about it.

But when I review the json I don't see a box for the person just a box for the snapshot and one for the face. I understand that face is a sub label.

So here for the after section I see:-

• a snapshot in which there is only a face scoring 55.46%
• the snapshot itself gets 80.8% score (but what does this actually mean)
• there's a label of person in the after section 95.7%
• the after section also has a top_score of 95.7% - again what does this actually mean
• it also has an attribute with face at 85.9%
• I would expect the snapshot to have a person label with a score

Any suggestions? Are there rules for how to interpret this hierarchy?

{
   "time":"2024-12-01T10:21:48.566338",
   "topic":"frigate/events",
   "message":{
      "before":{
         "id":"1733048478.073581-rfq86x",
         "camera":"front_door",
         "frame_time":"2024-12-01T10:21:42.815263",
         "snapshot":{
            "frame_time":"2024-12-01T10:21:32.976522",
            "box":[
               319,
               476,
               537,
               719
            ],
            "area":52974,
            "region":[
               300,
               332,
               688,
               720
            ],
            "score":0.80859375,
            "attributes":[
               {
                  "label":"face",
                  "score":0.5546875,
                  "box":[
                     330,
                     510,
                     398,
                     602
                  ]
               }
            ]
         },
         "label":"person",
         "sub_label":null,
         "top_score":0.95703125,
         "false_positive":false,
         "start_time":"2024-12-01T10:21:18.073581",
         "end_time":null,
         "score":0.93359375,
         "box":[
            226,
            537,
            449,
            716
         ],
         "area":39917,
         "ratio":1.2458100558659218,
         "region":[
            176,
            400,
            496,
            720
         ],
         "stationary":false,
         "motionless_count":8,
         "position_changes":1,
         "current_zones":[
            "porch"
         ],
         "entered_zones":[
            "driveway",
            "porch"
         ],
         "has_clip":true,
         "has_snapshot":true,
         "attributes":{
            "face":0.859375
         },
         "current_attributes":[
            
         ]
      },
      "after":{
         "id":"1733048478.073581-rfq86x",
         "camera":"front_door",
         "frame_time":"2024-12-01T10:21:46.807995",
         "snapshot":{
            "frame_time":"2024-12-01T10:21:32.976522",
            "box":[
               319,
               476,
               537,
               719
            ],
            "area":52974,
            "region":[
               300,
               332,
               688,
               720
            ],
            "score":0.80859375,
            "attributes":[
               {
                  "label":"face",
                  "score":0.5546875,
                  "box":[
                     330,
                     510,
                     398,
                     602
                  ]
               }
            ]
         },
         "label":"person",
         "sub_label":null,
         "top_score":0.95703125,
         "false_positive":false,
         "start_time":"2024-12-01T10:21:18.073581",
         "end_time":"2024-12-01T10:21:47.938235",
         "score":0.5546875,
         "box":[
            225,
            650,
            371,
            719
         ],
         "area":10074,
         "ratio":2.1159420289855073,
         "region":[
            50,
            400,
            370,
            720
         ],
         "stationary":false,
         "motionless_count":10,
         "position_changes":1,
         "current_zones":[
            "porch"
         ],
         "entered_zones":[
            "driveway",
            "porch"
         ],
         "has_clip":true,
         "has_snapshot":true,
         "attributes":{
            "face":0.859375
         },
         "current_attributes":[
            
         ]
      },
      "type":"end"
   }
}

[NickM-27]

There are two boxes in that payload, one for the person and one for the face. All of the fields you're questioning are describing the person

[blacknell]

1. Would you agree that the hierarchy isn't particularly obvious (to the uninitiated). One box is at same level as person (both are one level down in the snapshot level) and the other is within the attributes block (face).

2. How should I interpret the scores?

• 0.95703125 top_score that it's a person during this event
• 0.5546875 score that this snapshot is a person
• 0.859375 that this event has a face "face":0.859375
• 0.5546875 that this snapshot has a face

[NickM-27]

1. This is object level metadata so in general it is not meant to be obvious as most users are not interacting with this. It is snapshot because it is describing the object at the time that the best snapshot has been saved. All of the values under snapshot describe the object and then attributes describe the attributes of the object.

0.95703125 top_score that it's a person during this event

top_score is the highest median score of the object so far

0.5546875 score that this snapshot is a person

score is the score of the object in that snapshot

0.859375 that this event has a face "face":0.859375

this is the best scoring face during the history of the object

0.859375 that this event has a face "face":0.859375

this is the score of the face during the snapshot

[blacknell]

Ok. So I've got a snapshot in frigate where it's tagged with person, face, package. But when I then look at the event data behind that I can only see person and face. I'm not getting package - I would have thought it was an attribute?

I'm unclear on the difference between attributes and current_attributes

I was hoping I could see the list of attributes found during the detection event frames grow (with successive frames) and then react to the first time I see person with a package

Part of the difficulty is tracking down the event id from frigate UI eg. where can I see it was 1733057770.89996-yrsowh for example

[NickM-27]

You don't have a snapshot, because a snapshot can only have one object. Maybe you mean review item but if that's what you're looking at then you're using the wrong MQTT topic.

/events is for individual objects, /reviews is for activity

[blacknell]

Ah. So if I want to detect that a person with a package walked on the driveway do I need to look at reviews and then the detections referenced by it? Or can I just look at reviews

Do the before/after zones work in the same way as for events

I want to detect if a person with a package moved from the street to the driveway

[NickM-27]

Yes just look at reviews, the zones work in a similar way