Find function execution time based on start/end timestamps

Elastic Stack Kibana
1

Our ES index has timestamps, markers for function start and end (in separate documents) and other fields which identify execution environment (ip+pid+tid).

By fixing ip+pid+tid and calculating difference between timestamps of subsequent (end - start) I'll get execution time of function.

Are there any aggregation that looks to ordered series of documents and perform aggregation between two adjusted documents?

Like from:

{"ip": "192.168.0.1", "pid": "2345", "tid": "slave-3",
    "@timestamp": "2017-01-01 00:00:00",  "msg": "start calc"}
{"ip": "192.168.0.1", "pid": "2345", "tid": "slave-3
   ", "@timestamp": "2017-01-01 00:01:00",  "msg": "end calc"}
{"ip": "192.168.0.2", "pid": "1234", "tid": "slave-2",
   "@timestamp": "2017-01-01 00:00:02",  "msg": "start calc"}
{"ip": "192.168.0.2", "pid": "1234", "tid": "slave-2",
    "@timestamp": "2017-01-01 00:00:05",  "msg": "end calc"}

I can found that max execution time is 1 minute, and min execution time is 3 sec.

2

You might be able to use logstash to augment the documents with this information. Check the value of the msg field and add tags for start and end then use an elapsed filter.

3

Hi Gavenkoa,

I had the same issue as you - I have 'transaction start' and 'transaction end' events in a database that I wanted to work out the elapsed time of.

You should take a look at the elapsed-filter plugin. You basically configure it to watch for new events and you give it a field or combination of fields which would result in a unique ID with which to correlate the pair of start/end events. The elapsed filter then keeps a record of 'start' events and when it finds a matching 'end' event it calculates the time difference and inserts it as a new field in the end event (there are some other options).

As I say, it worked great for me.

Cheers,
Steve

4

Thanks for suggestion!

With your help I found some examples to start with:

Unfortunately I ingest data directly to ES without Logstash.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.