Calculating time between two events from same request id

[Mohammad_Alam]

I am new to Elastic and Kibana so any help here would be helpful.

I have logs coming in to elastic where a request with a unique id will go through various action stages, from request created to request approved to request granted. I am trying to figure out the best way to calculate time elapsed between states for each of the unique ids using the timestamp field. So for example request with id 1 comes in at 13:00:00, gets approved at 13:05:02 and granted at 13:08:24, what is the best way to calculate that and then visualize it?

[tsullivan (Tim Sullivan)]

It sounds like the documents in the Elasticsearch index pertain to timestamps that pertain to changes in "actions", and different documents are stored for the same individual action to track the change in the stage value. If so, then the data is "event-centeric" as it tracks events when things changed, and what changed in the event.

I recommend looking into the Transforms feature of Elasticsearch to pivot the data into a new index that is entity-centric around actions. Under that new design, each action will be a single document, with specific fields to track when the state changed. A transform is a continuous process that keeps the destination index up-to-date as new data comes in.

You can learn more about the Transforms feature here: https://www.elastic.co/guide/en/elasticsearch/reference/current/transform-overview.html

[Mohammad_Alam]

Thank you, i ended up doing exactly this. I created a transform that created one entry for an event with the start and end time as fields and then calculated the difference between the two via scripted field.

[tsullivan (Tim Sullivan)]

Glad to hear! Thanks for the follow-up.