How/Where to create graph with calculated field

Elastic Stack Kibana
1

Hi,
I'm trying to set up ELK as a PoC measure transaction performance. In particular to highlight when things are slow (several seconds).

So far I have logstash taking STDIN input and sending to elastic (I have another thread asking how to get filebeat to send the data for me). I added 7 documents this way.

If I go into visualise and set the timeframe back 12th Jan (when my sample log is generated) I can get the count of 7 records.

The data I'm feeding in from logstash looks like this;

{
       "@version" => "1",
     "BodyDetail" => "ENQUIRY.SELECT,,SOMEUSER//AU0010001,RB.CARD.VISA.CH.EXP,\r",
           "host" => "SYMV170150",
           "TxID" => "965fa342-43f1-4c72-a44b-c01670f414e2",
      "timestamp" => "2019-01-12 02:59:54.433",
       "LogLevel" => "Trace",
    "Description" => "Request",
      "LogSource" => "T24ServiceConnector",
        "message" => "2019-01-12 02:59:54.433 Trace    [T24ServiceConnector] Request [Tx 965fa342-43f1-4c72-a44b-c01670f414e2, MsgFormat OFS]: ENQUIRY.SELECT,,SOMEUSER//AU0010001,RB.CARD.VISA.CH.EXP,\r",
     "@timestamp" => 2019-01-11T15:59:54.433Z
}

{
       "@version" => "1",
     "BodyDetail" => "Unable to connect to the configured jagent instances due to unexpected error while processing the request.\r",
           "host" => "SYMV170150",
           "TxID" => "965fa342-43f1-4c72-a44b-c01670f414e2",
      "timestamp" => "2019-01-12 02:59:56.465",
       "LogLevel" => "Error",
    "Description" => "Error while processing request",
      "LogSource" => "T24ServiceConnector",
        "message" => "2019-01-12 02:59:56.465 Error    [T24ServiceConnector] Error while processing request [Tx 965fa342-43f1-4c72-a44b-c01670f414e2]: Unable to connect to the configured jagent instances due to unexpected error while processing the request.\r",
     "@timestamp" => 2019-01-11T15:59:56.465Z
}

What I think I want to be able to do, is look for the TxID and then get the difference between the oldest and newest.
The example transaction above failed, so maybe not the best example, but the timestamps show a difference of 32ms (2019-01-12 02:59:56.465 - 2019-01-12 02:59:54.433). It's this 32ms I want to graph against the latest entry time.
In reality, there are hundreds/thousands of transactions, so I'll probably want to filter to those taking more than 2 seconds.

I've been doing this laboriously in excel - so I've spent this week trying to learn how to get ELK running to hopefully save a heap of time in the future. If I need to change something in the other modules, no problem - this is PoC.
thanks in advance for pointers :slight_smile:

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.