- Version: 7.3.2
- Operating System: Ubuntu 18.04.2 LTS
Hi all,
I have a problem with failed to get json parse failure count in stats/pipelines. In my env, input comes from filebeat, when json parsing data failed due to incorrect input data format,I get json parseing error log(Error parsing json {:source=>"body", :raw=>"{"player_id":"2"u}", :exception=>#})。But when I request stats info(curl -XGET 'localhost:9600/_node/stats/pipelines?pretty'), there is no failures count. While I error occurs in other plugin(date, dissect), failure count works.
How can I get json parse failure count?
Thanks for your help!
logstash.conf:
input {
beats {
port => 5044
}
}
filter {
dissect {
mapping => {
"message" => "%{time} %{event} %{body}"
}
add_field => {
"[@metadata][message]" => "%{message}"
}
remove_field => ["message"]
}
date {
match => ["time", "UNIX"]
remove_field => ["time"]
}
json {
source => "body"
target => "body"
}
mutate {
rename => { "body" => "%{event}" }
}
}
output {
stdout {}
}
test input data:
1559102526 EnterGame {"player_id":"1"}
1559102543 EnterGame {"player_id":"3"}
1559102545 EnterGame {"player_id":"2"}
1559102526
dfadsfadsfadsfadfad
1559102545 EnterGame {"player_id":"2"}
155910dddd2545 EnterGame {"player_id":"2"}
155910dddd2545 EnterGame {"player_id":"2"u}
1559102545 EnterGame {"player_id"fdsafdasfsdf}:"2"}
response of _node/stats/pipelines?pretty
{
"host" : "ip-172-31-17-206",
...
"filters" : [ {
"id" : "0f585d5e4db725badb8b6e9f5a25c717f1c3986a96af0e8454d7ba75f0b1e293",
"failures" : 2,
"name" : "date",
"events" : {
"duration_in_millis" : 8,
"in" : 9,
"out" : 9
},
"matches" : 5
}, {
"id" : "04e7cfcbf5730702ffb82cc290051ef817a2c8a1e35559586d0366b698ea3447",
"name" : "json",
"events" : {
"duration_in_millis" : 63,
"in" : 9,
"out" : 9
}
}, {
"id" : "43c401ea2225daddf7125b009d98792111a30a9a2003306935c54e2072b928eb",
"failures" : 2,
"name" : "dissect",
"events" : {
"duration_in_millis" : 67,
"in" : 9,
"out" : 9
},
...
}