Comparing date/time from two different documents

Elastic Stack Kibana
1

Hi,

The scenario is this:

  1. Same index
  2. Two different documents A and B
  3. Document A contains a log entry for event "Client Requests X"
  4. Document B contains a log entry for event "Client Request Accepted"

How do I calculate the time difference between these two timestamps, and ideally how do I visualize this?

Thanks!

2

Kibana cannot currently do that sorry.

You'd need to add that to the second document during ingest.

3

How about comparing two timestamps aggregated over a specific term or field?

4

You can do it if it's in the same document, but not between documents. That currently needs to be done outside of the stack, in your own client for eg.

5

Ok, that basically rules out any time comparison in our setup where independent services on the network are logging to their individual log files and they are being sourced via Logstash into Elastic. I thought that was the whole idea with the Elastic Stack...or am I missing something? :slight_smile:

6

Are you looking to find discrepancies in response times?

7

Well, yes...sort of...I want to visualize the latency trend of a given business operation, and that latency is the delta between start of operation A on server A and end of operation B on server B. Both servers A and B's log files are in the same index, but obviously as two separate documents. They do, however, correlate via a unique transaction id which is an indexed field in both documents.

8

Yeah ok. There's no algorithm in Elasticsearch to calculate this, and Kibana doesn't have anything at this stage. If it were in the same event, eg start + end time, then you can add it with a scripted field.

Generally though it would be stored as a value in the event, eg as captured by Packetbeat or as calculated by Logstash.

9

For this type of analysis, you may want to consider entity-centric indexing.

1 Like
10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.