I have a lists of system logs in a log file ,in which I have to calculate time difference between two particular logs based on some condition
@timestamp message
May 19th 2016, 02:55:29.003 just some log
May 19th 2016, 02:55:29.003 retired 2 times
May 19th 2016, 02:55:29.200 System1 down -------
May 19th 2016, 02:55:29.205 just some log
May 19th 2016, 02:55:29.453 System1 up ----
May 19th 2016, 02:55:39.200 System2 down
May 19th 2016, 02:55:49.205 just some log
May 19th 2016, 02:55:53.453 System2 up -------
May 19th 2016, 02:55:58.453 all done
For example in the mentioned log, we need to trigger the calculation when we see the message 'retried' in some log.
On seeing the message it should start the calculating the difference from immediately followed message which contains 'System1 Down' message till it sees the 'System1 Up' message.But here the issue is that the system message can be dynamic like either 'System1' or 'System2'
If it starts with 'System1 Up' , it should calculate time till 'System1 Down' and same for 'System2 Up'. Is there a possibility to use regex in logstash.
And all the process should stop when it see 'all done' in the message. The time difference calculation should start again if it sees 'retired 2 times' in any of the logs that come below afterwards.
I tired to use elapsed plugin in Logstash and aggregate but not sure, it didn't work for this kinda flow.
What is best to do this? Either in Logstash , elastic search or Kibana. Thanks in advance