Comparing date/time from two different documents


#1

Hi,

The scenario is this:

  1. Same index
  2. Two different documents A and B
  3. Document A contains a log entry for event "Client Requests X"
  4. Document B contains a log entry for event "Client Request Accepted"

How do I calculate the time difference between these two timestamps, and ideally how do I visualize this?

Thanks!


(Mark Walkom) #2

Kibana cannot currently do that sorry.

You'd need to add that to the second document during ingest.


#3

How about comparing two timestamps aggregated over a specific term or field?


(Mark Walkom) #4

You can do it if it's in the same document, but not between documents. That currently needs to be done outside of the stack, in your own client for eg.


#5

Ok, that basically rules out any time comparison in our setup where independent services on the network are logging to their individual log files and they are being sourced via Logstash into Elastic. I thought that was the whole idea with the Elastic Stack...or am I missing something? :slight_smile:


(Mark Walkom) #6

Are you looking to find discrepancies in response times?


#7

Well, yes...sort of...I want to visualize the latency trend of a given business operation, and that latency is the delta between start of operation A on server A and end of operation B on server B. Both servers A and B's log files are in the same index, but obviously as two separate documents. They do, however, correlate via a unique transaction id which is an indexed field in both documents.


(Mark Walkom) #8

Yeah ok. There's no algorithm in Elasticsearch to calculate this, and Kibana doesn't have anything at this stage. If it were in the same event, eg start + end time, then you can add it with a scripted field.

Generally though it would be stored as a value in the event, eg as captured by Packetbeat or as calculated by Logstash.


(Christian Dahlqvist) #9

For this type of analysis, you may want to consider entity-centric indexing.


(system) #10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.