Grafana Webhook Integration: Looks for `tags` in the wrong part of the json payload

Issue Summary Grafana webhook doesn’t convert the tags sent by Grafana into custom attributes. This appears to be because in Grafana.py, it’s looking in the wrong part of the incoming json document for the tags.

Here’s the incoming alert payload from Grafana version 6.3.6, which is the newest version:

{ 
   "evalMatches":[ 
      { 
         "value":323.4,
         "metric":"samples",
         "tags":{ 
         }
      }
   ],
   "message":"Here's my message",
   "ruleId":12,
   "ruleName":"Samples Appended alert",
   "ruleUrl":"http://grafana.monitor.aort.<...>.com/d/itBdm3dZz/prometheus-health?fullscreen\\u0026edit\\u0026tab=alert\\u0026panelId=3\\u0026orgId=1",
   "state":"alerting",
   "tags":{ 
      "arbitrary_key":"arbitrary_val",
      "dashboard":"my dashboard",
      "slack":"my_slack",
      "tsg":"my tsg"
   },
   "title":"[Alerting] Samples Appended alert"
}

Here’s the Grafana alert config:

In Alerta’s grafana code (https://github.com/alerta/alerta/blob/master/alerta/webhooks/grafana.py#L33) it appears to be building custom attributes from the tag data from some match stanza rather than the top-level tags stanza where the sent tags are actually located.

[n0guest]

The thing I’m worried about is that if I need to set a specific environment on an alert it will be more work tagging all individual alert than maintaining a couple of webhook url’s.

Yep, it’s “the other side of the coin”. By combining both ways we’re providing end users a choice how to use it in their own specific case. The only consequence of such decision - complexity of logic in code. But as they say - “there is always a trade off”.

The thing I’m worried about is that if I need to set a specific environment on an alert it will be more work tagging all individual alert than maintaining a couple of webhook url’s.

As I wrote above, having both scenarios will allow you or anyone else to use the “better” for them - by handling the list of URL’s (with overrided params as query args) or by specifying addition custom tags in specific alert at Grafana (to override the same params).

So, @satterly, what do you think on that matter?

[psyhomb]

I’ve created a PR to address this issue ⬆️, so @satterly when you find time could you please review it and see if there is anything else that has to be done.

Thanks