Grokparsefailure tag - what does really it mean?

Hello,

I have next problem.
Started to use Filter->Grok in my Logstash recently and at the beginning everything was ok.
One day I noticed that to all messages from Logstash to Elastic... Logstash started to add a tag="[0] "_grokparsefailure"".
EVEN for configurations without "filter grok" module at all.

Here is one of examples which I really need to fix:

maxim@maxim-server2:/etc/logstash/conf.d$ sudo cat test.conf

input {
file {
path => "/var/log/test.log" ## <--- I have data in this format: 0% 29.169/36.729/44.274/3.952"
start_position => "beginning"
}
}
filter {

grok {
match => { "message" => "%{NUMBER:pinglosses}% %{NUMBER:pingmin}/%{NUMBER:pingavg}/%{NUMBER:pingmax}/%{NUMBER:pingmdev}"}
}

mutate {
convert => { "pinglosses" => "integer" }
convert => { "pingavg" => "integer" }
convert => { "pingmax" => "integer" }
convert => { "pingmin" => "integer" }
convert => { "pingmdev" => "integer" }
}

}
output {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "logstash3-%{+YYYY.MM.dd}"
document_type => "my_test3_doc"

}
stdout {}
}

In the log i always see:
Dec 10 16:34:22 maxim-server2 logstash[4414]: {
Dec 10 16:34:22 maxim-server2 logstash[4414]: "message" => "0% 32.746/36.992/42.751/2.560",
Dec 10 16:34:22 maxim-server2 logstash[4414]: "path" => "/var/log/test.log",
Dec 10 16:34:22 maxim-server2 logstash[4414]: "host" => "maxim-server2",
Dec 10 16:34:22 maxim-server2 logstash[4414]: "pingavg" => 36,
Dec 10 16:34:22 maxim-server2 logstash[4414]: "pingmax" => 42,
Dec 10 16:34:22 maxim-server2 logstash[4414]: "@timestamp" => 2018-12-11T00:34:21.999Z,
Dec 10 16:34:22 maxim-server2 logstash[4414]: "tags" => [
Dec 10 16:34:22 maxim-server2 logstash[4414]: [0] "_grokparsefailure"
Dec 10 16:34:22 maxim-server2 logstash[4414]: ],
Dec 10 16:34:22 maxim-server2 logstash[4414]: "pingmdev" => 2,
Dec 10 16:34:22 maxim-server2 logstash[4414]: "pingmin" => 32,
Dec 10 16:34:22 maxim-server2 logstash[4414]: "pinglosses" => 0,
Dec 10 16:34:22 maxim-server2 logstash[4414]: "@version" => "1"
Dec 10 16:34:22 maxim-server2 logstash[4414]: }

Actually I get all needed fields and values correct but what this "grokparsefailure" tag means?

IMPORTANT:
The test.conf locates at /etc/logstash/conf.d/ folder and starts automatically...
If I do the same conf starting manually...
logstash -f /etc/logstash/conf.d/test.log
I do NOT receive this grokparsefailure TAG.

Please help.

I just figured out the answer.

If you keep few logstash configuration files in the same directory and one of them has "grok filter" then all other conf files also will be using the same filter.
Just create the one conf file with multiple input/output and that's it.

3 Likes

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.