Logstash using date filter

[Guillaume_B (Guillaume)]

Hello everyone,

I am using Logstash to parsed the message field in 3 different fields. I am trying to use the date filter to set the type of one of my field to the date type.

This is an example of a line from my log file : " string:2017-10-20 17:45:04,020;string "

This is my logstash conf setting:

filter {
       dissect {
              mapping => { message => "%{field1}:%{field2};%{field3}" } 
       }
       date {
              match => [ "field2" , "yyyy-MM-dd HH:mm:ss:SSS" ]
       }
}           

The log file is parsed correctly in KIBANA but my fields all have the string type.

error message in logstash :

[2017-10-31T12:01:51,783][ERROR][logstash.shutdownwatcher ] The shutdown processappears to be stalled due to busy or blocked plugins. Check the logs for more information.

What am I doing wrong ?

Thanks in advance

[magnusbaeck (Magnus Bäck)]

Your date filter is failing because the timestamp you want to parse doesn't match your date pattern. Pay close attention to the character just before the milliseconds.

[Guillaume_B (Guillaume)]

You are right but when I replace the ":" with a "," to make the timestamp match to the date pattern it does not work. The Logstash error message stay the same but the new data isn't upload on KIBANA and ElasticSearch is not reporting anything.

My new date filter:

date {
      match => [ "field2" , "yyyy-MM-dd HH:mm:ss,SSS" ]
}

Thank you for your help !

[magnusbaeck (Magnus Bäck)]

I'm sure Logstash logs more messages than the one you've reported.