Logstash date filter not work

Elastic Stack Logstash
1

I try to use logstash date filter but it not work.

My conf is:
input {
stdin{}
}

filter {
date {
match => ["when", "YYYY-MM-dd HH:mm:ss,SSS"]
}
grok {
match => { "message" => "\ATID:%{SPACE}[%{INT:id}]%{SPACE}[(?([a-zA-Z0-9]))]%{SPACE}[%{TIMESTAMP_ISO8601:when}]%{SPACE}(?<log_level>([a-zA-Z]))\Z" }
}
}

output {
stdout {
codec => rubydebug
}
}

and I tested it in command file as:

echo "TID: [-1234] [] [2018-02-05 11:44:31,547] INFO" | /opt/logstash/bin/logstash -f /etc/logstash/conf.d/wso2.conf

I get this:
Logstash startup completed
{
"message" => "TID: [-1234] [] [2018-02-05 11:44:31,547] INFO",
"@version" => "1",
"@timestamp" => "2018-02-05T13:06:08.946Z",
"host" => "..................",
"id" => "-1234",
"when" => "2018-02-05 11:44:31,547",
"log_level" => "INFO"
}
Logstash shutdown completed

As you can see "@timestamp" is not replaced with "when".

Any idea?

Thanks

2

Filters are evaluated in order. Your date filter must come after the grok filter.

3

Thank you very much. Now it works :grinning:

My new conf is:
input {
stdin{}
}

filter {
grok {
match => { "message" => "\ATID:%{SPACE}[%{INT:id}]%{SPACE}[(?([a-zA-Z0-9]))]%{SPACE}[%{TIMESTAMP_ISO8601:when}]%{SPACE}(?<log_level>([a-zA-Z]))\Z" }
}
date {
match => ["when", "YYYY-MM-dd HH:mm:ss,SSS"]
}
}

output {
stdout {
codec => rubydebug
}
}

and the results of:
echo "TID: [-1234] [] [2018-02-05 11:44:32,548] INFO" | /opt/logstash/bin/logstash -f /etc/logstash/conf.d/wso2.conf

is:
Settings: Default pipeline workers: 2
Logstash startup completed
{
"message" => "TID: [-1234] [] [2018-02-05 11:44:32,548] INFO",
"@version" => "1",
"@timestamp" => "2018-02-05T09:44:32.548Z",
"host" => "........",
"id" => "-1234",
"when" => "2018-02-05 11:44:32,548",
"log_level" => "INFO"
}
Logstash shutdown completed

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.