Parse cloud flare logs in json format with ELK

I'm trying to create a logstash filter to parse the cloud flare json logs like this


using filebeat as input
input_type: log
json.keys_under_root: true
json.overwrite_keys: false
json.add_error_key: true

and logstash filter like this
filter {
date {
match => ["timestamp", "yyyy-MM-dd HH:mm:ss,SSS", "yyyy-MM-dd HH:mm:ss", "ISO8601"]
target => "@timestamp"
remove_field => "timestamp"
mutate {
gsub => [
"message", '"{', "'{",
"message", '}"', "}'"
kv {
source => "message"
remove_field => ["message"]
field_split => ","
value_split => ":"
trim_key => " "
trim_value => " "

But Kibana doesn't show the fields splitted correctly
In particular the first and the last.
Could you help me ?

Don't parse JSON with mutate and kv filters. Use a json filter or json codec in Logstash or enable Filebeat's JSON parser (which it seems like you've done).

So you suggest me to remove completely the logstash filter above and all works fine?

I've only added on 02-beats.conf the codec json
codec => "json"


This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.