Dear All,
I am currently learning how to parse data with logstash and pass them to Logstash. Right now I am confused on how to parse date correctly, as you can see, I have a timestamp with the format
yyyy/MM/dd HH:mm:ss.SS
Right now I am using the following for getting the timestamp, which works(but with a minor issue)
filter {
grok {
match => { "message" => "%{DATESTAMP:event_time},%{NUMBER:coord1},%{NUMBER:coord2},%{NUMBER:depth},%{NUMBER:magnitude},%{WORD:magtype}" }
}
mutate {
convert => [ "event_time", "string" ]
}
date {
locale => "en"
match => ["event_time", "yyyy/MM/dd HH:mm:ss'.'SS"]
target => "@timestamp"
remove_field => [ "timestamp" ]
add_field => { "debug" => "timestampMatched"}
}
}
What happened was that, after I parse my data with this configuration, a date like "2016/11/14" became "0016/11/14".
As we can see here, the DATESTAMP grok pattern consists of YEAR, MONTHNUM and other grok patterns. The YEAR grok pattern contains only 2 digits, which I think is the source of this problem, but I have no idea how to overcome it.
IF I didn't understand wrongly, what I need to do in the Logstash config is to extract the event_time from the log message with grok, then use mutate to change it into a string, and extract the date time pattern with "date" . And now I think the problem is I need to take the whole date stamp including all the four digits of the Year parameter, but I am not sure how to do that. It seems like grok pattern of YEAR only comes with 2 digit.
*I have tried using regex in grok pattern, but it doesn't work either.