Calculating time between specific logs based on condition

I have a lists of system logs in a log file ,in which I have to calculate time difference between two particular logs based on some condition

@timestamp                      message  

May 19th 2016, 02:55:29.003     just some log
May 19th 2016, 02:55:29.003     retired 2 times
May 19th 2016, 02:55:29.200     System1 down -------
May 19th 2016, 02:55:29.205     just some log  
May 19th 2016, 02:55:29.453     System1 up ----
May 19th 2016, 02:55:39.200     System2 down 
May 19th 2016, 02:55:49.205     just some log  
May 19th 2016, 02:55:53.453     System2 up  -------
May 19th 2016, 02:55:58.453     all done

For example in the mentioned log, we need to trigger the calculation when we see the message 'retried' in some log.

On seeing the message it should start the calculating the difference from immediately followed message which contains 'System1 Down' message till it sees the 'System1 Up' message.But here the issue is that the system message can be dynamic like either 'System1' or 'System2'

If it starts with 'System1 Up' , it should calculate time till 'System1 Down' and same for 'System2 Up'. Is there a possibility to use regex in logstash.

And all the process should stop when it see 'all done' in the message. The time difference calculation should start again if it sees 'retired 2 times' in any of the logs that come below afterwards.

I tired to use elapsed plugin in Logstash and aggregate but not sure, it didn't work for this kinda flow.

What is best to do this? Either in Logstash , elastic search or Kibana. Thanks in advance

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.