Filter out events by JSON content


(Federico) #1

Hello there,
I am running Filebeat on K8S and want to index just application files encoded in JSON which contain a field "classtype" with value "application". It seems a quite easy requirement, after trying all the possible combinations of json.message_key, include_lines and decode_json_fields I still didn't succeed in achieving what I wanted.. Can you please help me?

My extract from https://github.com/elastic/beats/blob/master/deploy/kubernetes/filebeat-kubernetes.yaml looks now like this

  kubernetes.yml: |-
- type: docker
  containers.ids:
  - "*"
  #      json.message_key: message
  #      json.keys_under_root: true
  #      json.overwrite_keys: true
  tail_files: true
  include_lines: ['"classtype":"application"']
  processors:
    - add_kubernetes_metadata:
        in_cluster: true
    - decode_json_fields:
        fields: ["message"]
        target: ""
  #            overwrite_keys: true

Thanks in advance!
Regards


(Shaunak Kashyap) #2

What about using the decode_json_fields processor like you are, and then after that, using the drop_event processor with an appropriate when condition: https://www.elastic.co/guide/en/beats/filebeat/current/drop-event.html?