Reduce the metadata generated by filebeat while pulling logs in Kubernetes

I am using filebeat(v 8.1.0) to pull logs from microservices deployed in kubernetes env. Right now, there is a huge amount of meta data generated by filebeat apart from the actual log message. Is there a way cut it sort to only log message and the source pod/app?

filebeat.autodiscover:
      providers:
        - type: kubernetes
          node: ${NODE_NAME}
          templates:
            - condition:
                contains:
                  kubernetes.container.name: "my-api"
              config:
                - type: container
                  paths:
                    - "/var/log/containers/*-${data.kubernetes.container.id}.log"
                  json.keys_under_root: true
                  json.message_key: message

    processors:
      - drop_event:
          when:
            contains:
              message: "/health_check HTTP/"
      - drop_fields:
          fields: ["kubernetes.labels.pod-template-hash", "kubernetes.replicaset.name", "kubernetes.node", "log","ecs", "agent.type", "agent.hostname", "agent.ephemeral_id", "agent.id", "kubernetes.namespace_uid", "kubernetes.namespace_labels"]

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.