Automation firing documents cause errors with Elasticsearch failures

Environment Home-Assistant version: 0.108.3 Elasticsearch version: 7.6.2

Relevant configuration.yml settings:

elastic:
  url: http://localhost:9200
  only_publish_changed: true
  publish_frequency: 60
  ilm_enabled: true
  ilm_policy_name: home-assistant
  ilm_max_size: 3gb
  ilm_delete_after: 14d
  exclude:
    domains: ['group', 'input_select', 'input_text', 'media_player', 'remote', 'script', 'timer', 'zone']  ...

Describe the bug Automations that fired in Home Assistant cause this error below:

Apr 25 16:22:25 UBUNTU64 hass[58789]: elasticsearch.helpers.BulkIndexError: ('2 document(s) failed to index.', [{'index': {'_index': 'hass-events-v4-000001', '_type': '_doc', '_id': 'bKKms3EBuGdnIacmL8ip', 'status': 400, 'error': {'type': 'mapper_parsing_exception', 'reason': "failed to parse field [hass.attributes.id] of type [long] in document with id 'bKKms3EBuGdnIacmL8ip'. Preview of field's value: 'office_light_on_when_motion'", 'caused_by': {'type': 'illegal_argument_exception', 'reason': 'For input string: "office_light_on_when_motion"'}}, 'data': {'hass.domain': 'automation', 'hass.object_id': 'office_light_on_if_motion_darkness_off', 'hass.object_id_lower': 'office_light_on_if_motion_darkness_off', 'hass.entity_id': 'automation.office_light_on_if_motion_darkness_off', 'hass.entity_id_lower': 'automation.office_light_on_if_motion_darkness_off', 'hass.attributes': {'last_triggered': datetime.datetime(2020, 4, 25, 23, 21, 33, 796462, tzinfo=<UTC>), 'id': 'office_light_on_when_motion', 'friendly_name': 'Office Light On If Motion & Darkness Off'}, 'hass.value': 1, '@timestamp': datetime.datetime(2020, 4, 25, 23, 21, 33, 796820, tzinfo=<UTC>), 'agent.name': 'My Home Assistant', 'agent.type': 'hass', 'agent.version': '0.108.3', 'ecs.version': '1.0.0', 'host.geo.location': {'lat': 000.000, 'lon': -000.000}, 'host.architecture': 'x86_64', 'host.os.name': 'Linux', 'host.hostname': 'UBUNTU64', 'tags': ['hass']}}}, {'index': {'_index': 'hass-events-v4-000001', '_type': '_doc', '_id': 'gKKms3EBuGdnIacmL8ip', 'status': 400, 'error': {'type': 'mapper_parsing_exception', 'reason': "failed to parse field [hass.attributes.id] of type [long] in document with id 'gKKms3EBuGdnIacmL8ip'. Preview of field's value: 'office_light_off_if_office_darkness_on'", 'caused_by': {'type': 'illegal_argument_exception', 'reason': 'For input string: "office_light_off_if_office_darkness_on"'}}, 'data': {'hass.domain': 'automation', 'hass.object_id': 'office_light_off_if_office_darkness_on', 'hass.object_id_lower': 'office_light_off_if_office_darkness_on', 'hass.entity_id': 'automation.office_light_off_if_office_darkness_on', 'hass.entity_id_lower': 'automation.office_light_off_if_office_darkness_on', 'hass.attributes': {'last_triggered': datetime.datetime(2020, 4, 25, 23, 21, 40, 588801, tzinfo=<UTC>), 'id': 'office_light_off_if_office_darkness_on', 'friendly_name': 'Office Light Off If Office Darkness On'}, 'hass.value': 1, '@timestamp': datetime.datetime(2020, 4, 25, 23, 21, 40, 589076, tzinfo=<UTC>), 'agent.name': 'My Home Assistant', 'agent.type': 'hass', 'agent.version': '0.108.3', 'ecs.version': '1.0.0', 'host.geo.location': {'lat': 000.000, 'lon': -000.000}, 'host.architecture': 'x86_64', 'host.os.name': 'Linux', 'host.hostname': 'UBUNTU64', 'tags': ['hass']}}}])

To Reproduce Steps to reproduce the behavior:

1. Run journalctl -f to start seeing detailed HA logging
2. Cause an automation to fire in HA
3. Wait for this error to appear

Expected behavior Successfully index these events and avoid errors to ES.

Thank you!

[JZ-SmartThings]

@legrego do you even have to ask? 😃 I’d be delighted to and would appreciate it quite a bit…

[JZ-SmartThings]

Of course willing to share @legrego… took me much less time to grab screenshots than for you to develop this lovely piece of work.

The first screenshot is all me and leveraging SmartThings > MQTT bridge > Node-RED > finally Node-RED posts to Elasticsearch. Had this main dashboard for like 2 years.

The second screenshot is based on your work and it’s my quick and dirty that I’ve been looking for with respect to HA automation triggering (used Kibana Lens for the first time, not bad).

Lastly, the bottom most screenshot is my other quick and dirty in Grafana — which I just started using due to this Automation firing desire — and wanted to see what Grafana has to offer. Been playing with Grafana for a couple of weeks and mixing Elastic & MariaDB (any SQL query I desire on a chart) is an awesome feature Kibana lacks. Thanks again for everything, very cool addition to my little castle 😃