Date format issue for two different log types into same datefield in logstash

Elastic Stack Logstash
1

I have a common timedate field named TIME_END for two different date format for two different log types ,, in the first format it comes in UNIX type eg : 1460555937090 (millisecond+last 3 digits in nanoseconds) and in case of second log format it comes like this.. 2016-05-13 11:22:28,398

So in Case of first fomrat i have specified

if[logtype]=="1" {
    	date {
    	    match => [ "Time_End","UNIX" ] // UNIX_MS in case of windows
    	    target => "Time_End"
        
    	}
    }

if[logtype]=="2" {

date {

      match => [ "Time_End", "ISO8601", "YYYY-MM-dd HH:mm:Ss,SSS" ]

      target => "Time_End"

      locale => "en"

    }

i am bit confused here... sometimes i get an error in first format and sometimes in other.. can you please suggest me what should be my correct date condition for both..currently its going as string hence Kibana not able to use it as a timestamp. So i need help badly @ logstash end.

1 Like
2

We are also facing similar issue. When we have different time formats in two logs , how to convert them to a common format ?

4 
  match => [ "Time_End", "ISO8601", "YYYY-MM-dd HH:mm:Ss,SSS" ]

"Ss,SSS" should be "ss,SSS".

i am bit confused here... sometimes i get an error in first format and sometimes in other..

What error? Please be specific.

can you please suggest me what should be my correct date condition for both..currently its going as string hence Kibana not able to use it as a timestamp. So i need help badly @ logstash end.

Use an index template to make sure the the Time_End field is mapping as a timestamp. Look at how the @timestamp field is mapped in Logstash's default index template.