Is there any way to remove log.syslog.structured_data

Elastic Stack Logstash
1

I am trying to parse checkpoint and fortinet logs through logstash and get log.syslog.structured_data field in kibana/logscale .Is there any way to remove it ? I know it represents structured data expressed in RFC 5424 messages but I want to exclude that field completely as it's a complete replica of rawstring

2

Welcome to the community!

If you want to delete the field:
mutate{ remove_field => [ "[log][syslog][structured_data]"] }

If there is nothing important in all "log" nested field/structure, simply delete:
mutate{ remove_field => [ "log"] } # this is an array, can have multiple fields

You can also test on the "event" field.

3

Thanks for your reply ,I tried doing that but it didn't work .Here is the filter I am applying

filter {

 grok {
      match => {"message" => "%{SYSLOG5424PRI}%{GREEDYDATA:message}" }
      overwrite => [ "message" ]
    }

 mutate {
    remove_field => ["tags", "input", "log", "event", "@timestamp", "agent", "ecs", "@version", "host"]
  }

 kv {
    field_split => " "
    value_split => "="
  }

 mutate {
    remove_field => ["message"]
    rename => { "<(\d+)>date" => "date" }
    add_field => { "logdate" => "%{date} %{time}"}
  }

 date {
 
    match => ["logdate", "yyyy-MM-dd HH:mm:ss" ]
    timezone => "Europe/London"
    target => "@timestamp"
  }

 mutate {
 
    remove_field => ["logdate", "date", "time"]
    convert => { "rcvdbyte" => "integer"}
    convert => { "sentbyte" => "integer"}
 
  }
  mutate{ remove_field => [ "[log][syslog][structured_data]"] }
 
}
}
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.