Drop logs on the basis _jsonparsefailure


(Bharat Gupta) #1

"tags": [
"beats_input_codec_json_applied",
"_jsonparsefailure"
]

i want drop the log detail if get _jsonparsefailure

i have applied this inside filter :-

if "_jsonparsefailure" in [tags]{
drop { }
}

but still not resolving my problem i think it will apply only if single value "_jsonparsefailure" inside the tags but in my case getting 2 values "beats_input_codec_json_applied", "_jsonparsefailure"

give me appropriate solution


(Magnus Bäck) #2

but still not resolving my problem i think it will apply only if single value "_jsonparsefailure" inside the tags

No, not true.

The little piece of your configuration that you've showed us looks fine. Please show us the full configuration and an example document that should've been dropped but wasn't. Copy/paste from Kibana's JSON tab or use a stdout { codec => rubydebug } output to dump the raw event.


(Bharat Gupta) #3

input {
beats {
port => 5044
codec => json
}
}

filter {

if [tags] == ["beats_input_codec_json_applied","_jsonparsefailure"] {
drop { }
}
json {
source => message
}
date
{
match => ["timestamp" , "ISO8601", "yyyy-MM-dd HH:mm:ss,SSS"]
target => "@timestamp"
}
mutate {
remove_field => [message,json]
}

}

output {
elasticsearch {
hosts => "localhost:9200"
index => "demo-%{+YYYY.MM.dd}"
document_type => log
}
stdout { codec => rubydebug }
}


(Magnus Bäck) #4

Is it the json codec or the json filter that adds the _jsonparsefailure tag?


(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.