Support for event.ingested and event.processed
See original GitHub issueevent.created
is described as the time when the event was first read by an agent or by a pipeline. In a distributed setup however that means that information is lost about the time taken to process an event at the various stages.
Would it be possible to add new fields to event
to keep that information? For example, event.processed
can hold the time when e.g. logstash (the last one in a chain or maybe an array) actually processes the event, and event.ingested
when an ingest pipeline handles it.
Does this make sense?
Issue Analytics
- State:
- Created 4 years ago
- Comments:12 (7 by maintainers)
Top Results From Across the Web
Event Fields | Elastic Common Schema (ECS) Reference [8.5]
The event fields are used for context information about the log or metric event itself. A log is defined as an event containing...
Read more >Event ingestion explained - Handbook - PostHog
This document gives an overview of how data ingestion… ... Decompressing and normalizing the shape of event data for the rest of the...
Read more >Event ingestion | Camunda Platform 8 Docs
The Event Ingestion REST API ingests business process related event data from any third-party system to Camunda Optimize. These events can then be ......
Read more >Ingest from event hub - Azure Data Explorer - Microsoft Learn
This article describes how to ingest data from Azure Event Hubs into Azure Data Explorer.
Read more >Ingested raw event types - IBM
Business Automation Insights can ingest specific types of raw events. Ingestion. The event types and the description of the raw events that can...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Not fully. We’ve merged and released
event.ingested
, but we haven’t addressed the pipeline timestamp yet. Let’s keep it open for now.It’s something we discussed in the past on goes a bit into event tracing. I can definitively see the value of these “measurements” on the event. I wonder if we could come up with a more generic way of storing multiple timestamps and we could let everyone add as many timestamps he needs.