I'm collecting logs from an application recording the start time. Then another record is signifying the end time. Both records have to match on DeviceId and a CorrelationId. I'm using an ingest pipeline to process the logs. When the log line is processed that is the end time, I want to query ES and get the starttime and calculate the difference, and add a field for it.
I'm at a loss on how and if this can be done in pipelines in Elasticsearch.
I saw something similar via Logstash, but I'm trying to avoid Logstash and just do this in Elasticsearch.