ECS Ingest processor

With ECS we know the exact structure of some fields. Based on this some common processing happens. A few examples:

source.ip -> geoip processor to enrich with geo information user_agent.original -> user_agent enrichement

Since Elasticsearch 6.5 it is possible to have a pipeline that calls an other pipeline: https://www.elastic.co/guide/en/elasticsearch/reference/6.5/pipeline-processor.html We could provide an ECS pipeline that does all this default processing. All that users would have to do is adding it to their ingest pipeline.

Over time we could add more processing in such a pipeline. For example if we have a convention that all http.request.method should be upper case the Uppercase Processor could be applied to this field and similar things.

[webmat]

• Create an ingest pipeline that converts GELF to ECS

[webmat]

• calculate a directional geo_shape when both ends of a network connection have a geo_point. See https://github.com/elastic/beats/issues/11702 for details