Kibana visualization for time difference

[Bikash_Behuria]

Hi ,

I have Request Response Integration logs with correlation ID in both request and response logs.

5c023e43-b695-407b-ac17-1f85d10ec9e4 RP401 OPT OrderId 11087766 (Request)
5c023e43-b695-407b-ac17-1f85d10ec9e4 RS401 OPT OrderId 11087766 (Response)

And elasticsearch timestamp field as well. And the first field is the CorrelationId and format JSON is like below

{
"_index": "applicationlogs",
"_type": "applicationlogs",
"_id": "AWEIM3fktvuk7rH85tyK",
"_version": 1,
"_score": null,
"_source": {
"@timestamp": "2018-01-18T07:37:26.210Z",
"system": "OPT",
"messagetype": "RS401",
"businessIdType": "OrderId",
"CorelationId": "5c023e43-b695-407b-ac17-1f85d10ec9e4",
"@version": "1",
"businessId": "11087766",
"message": "5c023e43-b695-407b-ac17-1f85d10ec9e4 RS401 OPT OrderId 11087766",
"type": "applicationlogs",
"tags": [
"_jsonparsefailure"
]
},
"fields": {
"@timestamp": [
1516261046210
]
},
"sort": [
1516261046210
]
}

Now my requirement is to find out the time difference between Request and Response and based on that , I want to create some Visualization.

Can someone please help ?

Thanks-Bikash

[lukas (Lukas Olson)]

I can't think of a way to do this without further enriching your data prior to indexing in Elasticsearch. Are you able to add a step prior to indexing that adds the time difference to the document corresponding to the response?

[Bikash_Behuria]

Hi ,

Since this is a request reply async pattern , my one process sending request , so I am logging the request time and another process receiving response , then I am logging the response time. The common field is the correlation Id , which I am sending in both logs. So it will be two log statement received in two different time.

My pipeline is like below

Application --> Kafka topic --> Logstash --> Elasticsearch --> Kibana

I am sending this logs via Logstash using Grok filter to parse . I was searching a solution using elasticsearch query , and then wanted to show the visulaization in Kibana.

Thanks-Bikash