Strugging with the date filter

[JonConnor]

Having a difficult time with the data filter. the 'message' field contains:

2018.06.27 00:22:23:209 UTC | FieldDebug | CommonPersistency

my logstash filter:

filter {
date {
match => ["message", "yyyy.MM.dd HH:mm:ss:SSS ZZZ"]
target => "@timestamp"
}
}

Tried different variations of the timezone ZZZ => no space between S and Z, single Z, not including it in the match). all results in _dateparsefailure.

Any advice on what i'm doing wrong ?

[Christian_Dahlqvist]

You need to first separate out the timestamp part into a separate field, e.g. using dissect or grok, and then run the date filter on this.

[JonConnor]

DOH...thanks wasn't sure if date was in place of grok.

Some advice on the grok filter then? I created a custom pattern:

DATESTAMP_LOGFILE %{YEAR}.%{MONTHNUM2}.%{MONTHDAY} %{TIME} %{TZ}

which is loaded ok when i start logstash. but getting grok_parse errors now:

logstash config:
filter {
grok {
patterns_dir => ["/app/custom.Grok.Patterns"]
match => ["message", "%{DATESTAMP_LOGFILE:log_timestamp"]
}

date {
match => ["log_timestamp", "yyyy.MM.dd HH:mm:ss:SSS ZZZ"]
target => "@timestamp"
}
}