Plotting time diff based on uniq thread_id and API

Elastic Stack Kibana
1

i have a log file as below. Fields are indexed as [thread_id] [API] [method] [timestamp] [GREEDYDATA] respectively.
describing more on first two lines ,
1st line --> 1111 is the thread_id for api=/API1 with init time (method=GET) as Feb 11 22:27:13 PST 2020 and
2nd line ---> Response to first line (method=RESP) with response timestamp as Wed Feb 11 22:27:15 PST 2020
my requirement is to calculate time diff (GET timestamp - RESP timestamp) for an API with unique thread id. Can someone please guide me on this.

1111 /API1 GET Wed Feb 11 22:27:13 PST 2020 XYZ
1111 /API1 RESP Wed Feb 11 22:27:15 PST 2020 status=200
1112 /API2 GET Wed Feb 11 22:28:13 PST 2020 XYZ
1112 /API2 RESP Wed Feb 11 22:28:15 PST 2020 status=200
1113 /API3 GET Wed Feb 11 22:29:13 PST 2020 XYZ
1113 /API3 RESP Wed Feb 11 22:30:15 PST 2020 status=200
2

hi Paresh,

You can try custom ruby code using ruby plugin , in logstash config.

Thanks
dilip

3

Thanks for the response Dilip,
you mean adding another event using logstash(and ruby) in ES ?
what will be the sample ruby code look like ?

4

hi Pranesh,

you can refer to the below for idea

Reagrds
dilip

5

in your case(and the original problem as well), start and end time was filtered and ingested in one go(and in one line). But in my case, these logs will be ingested live (app server logs), so at a given time, logstash will have access to only start time (API init call) and may be after 2-3 seconds it will have end time in another line. this make it very complex to calculate difference before ingesting to ES.
please correct me if i got it wrong. and if you have any solution which can work on already ingested data.

6

Hi Paresh

may be you can go with scripted fields in kibana, using painless script

Thanks
dilip

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.