Changing Time Format

[Wilks]

I have a filter that is reading 2 fields (STARTTIME and ENDTIME) from a CSV file and then changing to deviceCustomString1 and deviceCustomString2 and I need to change the time layout. I am trying to change the time format from say 7/6/2021 8:29:58 AM to epoch (MMM dd yyyy HH:mm:ss) can this be done?

filter {
 csv {
     separator => ","
     skip_header => "true"
     columns => ["STARTTIME","ENDTIME"]
      }

    rename    => {"STARTTIME"              => "deviceCustomDate1"}
    rename    => {"ENDTIME"                => "deviceCustomDate2"}
    
       }
       
}

 output { 
 stdout { codec => cef { reverse_mapping => false fields => [ "deviceCustomDate1", "deviceCustomDate2" ] } 
 } 
 }

So output looks like this but I need to change time to EPOCH time (MMM dd yyyy HH:mm:ss)

CEF:0|Elasticsearch|Logstash|1.0|Logstash|Logstash|6|deviceCustomDate1=7/2/2021 9:15:00 AM deviceCustomDate2=7/2/2021 6:06:00 PM

[Badger]

If you want those fields to be strings in a particular format then I suggest using a date filter to parse them into LogStash::Timestamp objects, then ruby and strftime to set the format you want. Something like this

date {
    match => [ "deviceCustomDate1", "M/d/YYYY h:mm:ss a" ]
    target => "[@metadata][deviceCustomDate1]"
}
ruby {
    code => '
        t = Time.at(event.get("[@metadata][deviceCustomDate1]").to_f)
        event.set("deviceCustomDate1", t.strftime("%b %d %Y %H:%M:%S"))
    '
}

Not sure if you want to replace %d with %-d or %e.

[Wilks]

Thanks this worked. The only issue I am having is sometimes there is no data in the deviceCustomDate1 and deviceCustomdate2 fields and when I add the above syntax it adds the below to the empty field. Is there a way to keep this empty when there is no data an not input Dec 31, 1969?

deviceCustomDate1=Dec 31 1969 19:00:00 deviceCustomDate2=Dec 31 1969 19:00:00

[Badger]

You could wrap the two filters in

if [deviceCustomDate1] {
    ...
}