Date Parsing ignored in Logstash-->@timestamp not updated


#1

I've got logs includiong timesamps like: "04/Nov/2016:00:00:16 +0100" I parse them via grok in a variable I call "apacheTime" and I want to use the date filter to parse them and overwrite "@timestamp".
So I included:

date {
locale => "en"
match => [ "apacheTimestamp", "dd/MMM/YYYY:HH:mm:ss Z" ]
}
as an additional filter.
However the @timestamp variable remains unchanged (and there is no error or anything).

Minimal example below:

Input:

99.99.99.99 - - [04/Nov/2016:07:00:00 +0100]

logstash .conf:

The # character at the beginning of a line indicates a comment. Use

comments to describe your configuration.

input {
beats {
port => "5043"
}
}

The filter part of this file is commented out to indicate that it is

optional.

filter {
grok {
patterns_dir => ["./patterns"]
match => { "message" => "%{IP:clientIp} %{APACHEDATA:remoteLogname} %{APACHEUSER:remoteUser} [%{HTTPDATE:apacheTimestamp}]"}
}
date {
locale => "en"
match => [ "apacheTimestamp", "dd/MMM/YYYY:HH:mm:ss Z" ]
}

}
output {
stdout {codec => rubydebug}

}

Pattern file:

APACHEREPSIZE %{NUMBER}|-
APACHEHOSTNAME %{HOSTNAME}|-
APACHEIP %{IP}|-
APACHEURI %{URI}|-
APACHEDATA %{DATA}|-
APACHESTRING %{QUOTEDSTRING}|-
APACHEUSER %{USER}|-
TIMEZONE [+|-]%{INT}
HTTPDATE %{MONTHDAY}/%{MONTH}/%{YEAR}:?%{HOUR}:?%{MINUTE}(:%{SECOND}) ?%{TIMEZONE}


(Magnus B├Ąck) #2

Please show an example event as produced by your stdout {codec => rubydebug} output.


#3

Problem magically solved itself (see screenshot for result).
I didn't change anything on the configuration since Friday. However a new JDK was installed on the testmaschine, maybe that solved the problem or the testmaschine just needed some rest at the weekend :smiley:


(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.