Find time difference between series of events with uniqueid

Elastic Stack Logstash
1

Hi ,

I have requirement where I have series of events that has a unique trace id. I want to find and take the timestamp from the first and last event of this id and find the difference.

Since I dont have any start and end tag to track for those events, i'm not able to use elpased filter. Is there a way with which I can achieve this requirement ?

Below is the sample log:

e65f915e-d3c4-471b-89eb-3613cd1f3c54 asd {"timestamp":"2020-10-29 08:18:48.893"}
e65f915e-d3c4-471b-89eb-3613cd1f3c54 def {"timestamp":"2020-10-29 08:18:49.111"}

2

You may be able to use an aggregate filter. See example 3 in the documentation.

An example of calculating time difference is here.

3

Thank you so much for your reply.

However for calculating the timedifference I should be able to get the timestamp of the first event and last event from the series of events of the same trace id. Does aggregate have option for this?

4

In the aggregate filter you would use some code like

code => '
    map["firstEvent"] ||=  event.get("@timestamp")
    map["lastEvent"] =  event.get("@timestamp")
'
timeout_code => '
    require "time"
    starttime = Time.iso8601(event.get("firstEvent").to_s).to_f
    endtime   = Time.iso8601(event.get("lastEvent").to_s).to_f
    event.set("overallTime", endtime - starttime)
'
5

Thanks very much, it worked!!!.

We also have requirement where on the same series of event instead of taking the timestamp for the last event we want to take the timestamp of a event in-between and find difference based on that.

For example we have the below series of events

65f915e-d3c4-471b-89eb-3613cd1f3c54 asd {"timestamp":"2020-10-29 08:18:48.893"}
e65f915e-d3c4-471b-89eb-3613cd1f3c54 def {"timestamp":"2020-10-29 08:18:49.111","status":"COMPLETED"}
e65f915e-d3c4-471b-89eb-3613cd1f3c54 def {"timestamp":"2020-10-29 08:18:49.111"}

So based on the key status is it possible to take the timestamp of that event and calculate ?

6

You could replace that with something like

if event.get("message").include?("COMPLETED")
    map["lastEvent"] =  event.get("@timestamp")
end

You may then need error handling if there are cases where that string does not occur, so lastEvent never gets set.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.