Hello,
I have next problem.
Started to use Filter->Grok in my Logstash recently and at the beginning everything was ok.
One day I noticed that to all messages from Logstash to Elastic... Logstash started to add a tag="[0] "_grokparsefailure"".
EVEN for configurations without "filter grok" module at all.
Here is one of examples which I really need to fix:
maxim@maxim-server2:/etc/logstash/conf.d$ sudo cat test.conf
input {
file {
path => "/var/log/test.log" ## <--- I have data in this format: 0% 29.169/36.729/44.274/3.952"
start_position => "beginning"
}
}
filter {
grok {
match => { "message" => "%{NUMBER:pinglosses}% %{NUMBER:pingmin}/%{NUMBER:pingavg}/%{NUMBER:pingmax}/%{NUMBER:pingmdev}"}
}
mutate {
convert => { "pinglosses" => "integer" }
convert => { "pingavg" => "integer" }
convert => { "pingmax" => "integer" }
convert => { "pingmin" => "integer" }
convert => { "pingmdev" => "integer" }
}
}
output {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "logstash3-%{+YYYY.MM.dd}"
document_type => "my_test3_doc"
}
stdout {}
}
In the log i always see:
Dec 10 16:34:22 maxim-server2 logstash[4414]: {
Dec 10 16:34:22 maxim-server2 logstash[4414]: "message" => "0% 32.746/36.992/42.751/2.560",
Dec 10 16:34:22 maxim-server2 logstash[4414]: "path" => "/var/log/test.log",
Dec 10 16:34:22 maxim-server2 logstash[4414]: "host" => "maxim-server2",
Dec 10 16:34:22 maxim-server2 logstash[4414]: "pingavg" => 36,
Dec 10 16:34:22 maxim-server2 logstash[4414]: "pingmax" => 42,
Dec 10 16:34:22 maxim-server2 logstash[4414]: "@timestamp" => 2018-12-11T00:34:21.999Z,
Dec 10 16:34:22 maxim-server2 logstash[4414]: "tags" => [
Dec 10 16:34:22 maxim-server2 logstash[4414]: [0] "_grokparsefailure"
Dec 10 16:34:22 maxim-server2 logstash[4414]: ],
Dec 10 16:34:22 maxim-server2 logstash[4414]: "pingmdev" => 2,
Dec 10 16:34:22 maxim-server2 logstash[4414]: "pingmin" => 32,
Dec 10 16:34:22 maxim-server2 logstash[4414]: "pinglosses" => 0,
Dec 10 16:34:22 maxim-server2 logstash[4414]: "@version" => "1"
Dec 10 16:34:22 maxim-server2 logstash[4414]: }
Actually I get all needed fields and values correct but what this "grokparsefailure" tag means?
IMPORTANT:
The test.conf locates at /etc/logstash/conf.d/ folder and starts automatically...
If I do the same conf starting manually...
logstash -f /etc/logstash/conf.d/test.log
I do NOT receive this grokparsefailure TAG.
Please help.