Calculate time difference between two dates ( start date and end date of an log file )

Hello, everyone.
I'm trying without success to calculate the difference between the start date and the end date of my log file in order to get the time duration ( in second ) of an script.

Here is my input :

17/04/2019  13:30:00 STARTED INFO :START OF THE LOG FILE
17/04/2019  13:35:00 ENDED INFO :END OF THE LOG FILE

What I want in output is :

{
start_date : 17/04/2019 13:30:37,
end_date : 17/04/2019 17:13:02,
log_duration : 300.00
}

And here is my conf file :

input {
  file {
    path => ["C:/Users/log.conf"]
    start_position => "beginning"
    sincedb_path => "NUL"
    codec => plain {
                   charset => "ISO-8859-1"
                  }
    
  }
}
filter {
  
  grok{
    
    patterns_dir => ["../patterns"]
    match =>{"message" => "%{DATE:date_date}%{SPACE}%{TIME:time}%{STATUS:state}%{SPACE}%{LOGLEVEL:level}%{SPACE}\:%{GREEDYDATA:msg}"}

  } 

if [state] =="STARTED" {

 mutate{
   add_field =>{
     "start_date" => "%{date_date} %{time}"
   }
  
 }
  date{
      match => ["start_date","dd/MM/yyyy HH:mm:ss"]
      target => "start_date"
    }
  
}
  
if [state] == "ENDED" {

    mutate{
      add_field => {
      "end_date" => "%{date_date} %{time}"
      }

    }
    date{
      match => ["end_date","dd/MM/yyyy HH:mm:ss"]
      target => "end_date"
    }
 

}   

ruby {
      
      code => 'event.set("duration_hrs", (event.get("end_date")- event.get("start_date")) ) rescue nil'
    }

}
mutate {
    remove_field => [ "date_date", "time","state","level","msg" ]
  }

output {
if "_grokparsefailure" not in [tags] {
  
    stdout { codec => rubydebug { metadata => true }}
}


}

Thank you in advance for helping me :+1:

Take a look at example 1 in the documentation of the aggregate filter.

thank you #Badger, i'll take a look at that

It works well, thanks:clap::clap:

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.