Introduce display_name for threat.indicator

Summary

Introduce Display Name for an Indicator of Compromise (IoC) in the Threat Intelligence part of ECS

Motivation:

In the Threat Intelligence capabilities of the Security Solution, our team is working on the data grid for IoCs (Indicator of Compromise) where we have an “Indicator” column, which serves as a “Display name” for an indicator. The value of this column currently depends on the indicator type and every type has its own logic. The best example is the File indicator. It can have different hashes, eg. sha256, md5, etc. We take sha256 for display name when available, if not then md5 and so on. For other types there is not much logic involved, we take different values from different threat.indicator* attributes per type. The problem with this approach is that this Display Name is not available as an attribute on Elasticsearch documents coming from Threat Intelligence integrations, therefore users won’t be able to filter by it and perform other standard operations they can do for existing ECS attributes. This can be solved partly with runtime fields, but we think adding Display Name to the schema might make sense and want to kick off the discussion about it Features dependant on having display_name field:

1. Filter in/out the Indicator of Compromise view by display_name
2. Add Indicator display_name to a Timeline
3. Create an Indicator of Compromise event renderer in Timelines
4. Create a pre-built Timeline template for investigating IoCs

Detailed Design:

Introduce threat.indicator.display_name with the following logic per threat.indicator.type

file: 
threat.indicator.file.hash.* (sha256 | md5 | sha1 | sha224 | sha3-224 | sha256 | sha3-256 | sha384 | sha3-384 | sha512 | sha3-512 | sha512/224 | sha512/256 | ssdeep | tlsh | impfuzzy | imphash | pehash | vhash)

url: 
threat.indicator.url.original

email-addr: 
threat.indicator.email.address

email (subj?): 
the suitable field is missing, map to _id?
 
email-message:
the suitable field is missing, map to _id?

domain-name:
threat.indicator.url.domain

domain:
threat.indicator.url.domain

ipv4-addr: 
threat.indicator.ip

ipv6-addr: 
threat.indicator.ip

x509-certificate:
threat.indicator.x509.serial_number

x509 Serial:
threat.indicator.x509.serial_number

windows-registry-key:
threat.indicator.registry.key

autonomous-system:
threat.indicator.as.number

mac-addr:
threat.indicator.mac

unknown:
map to _id

Data examples can be found in AbuseCH, Anomali, Cybersixgill, MISP, OTX, Recorded Future and ThreatQ integrations

[peasead]

I think that threat.indicator.name would align with other ECS fieldsets (host.name ,user.name, etc.)

Also, here’s the email ECS fieldset https://github.com/elastic/ecs/blob/main/schemas/email.yml, which could be nested under threat.indicator.

[maxcold]

@ebeahan btw what is usually the approach for backfilling the data if a new ECS field is introduced (or if there is another change in the schema)? Is there a common way to handle such cases? Specifically for threat.indicator.name it would be good to provide a way for users to add this field to the already existing data. Is it a good idea in general? how should we handle this?