In our actual logs, we sometimes need to calculate the elapsed time between two events.
We don't want to use the "elapsed" function in Logstash because it's not always needed and because elapsed time may ocasionnaly have to run during long periods waiting for the second event to appear.
I manage an MFT platform which purpose is to receive files from a source and pushing them to a specific target : source ==> Platform ==> target
The idea is to use our multiple date fields added in our events to calculate elapsed time between an entering file and the outgoing file.
Events are associated by a transfer_id, so a search request on that ID will give us the two corresponding events.
Events also have an identifier to know if it's an entering transfer or an outgoing transfer.
Each event has a "start_date" and an "end_date"
The idea is to calculate the duration between "start_date" of the first transfer and the "end_date" of the second transfer.
Do you think it's possible to do this easily in Kibana ?
The final goal is to obtain a time series visualisation, showing the evolution of the transfers duration from source to target (by joining the events together through their common transfer_id)
EDIT: If not possible by joining events on a common field, I might have a solution to have both start date of the first event and the end date of the second event, in an only one event.
But then I'll still have to find a way to calculate duration between the two dates.
Thanks for advices.