Filebeats not dropping agent.* fields

Elastic Stack Beats
1

We are using filebeats 7.4.0 in a k8s cluster to ship logs to ES, however when specifying a processor to drop the agent.* fields they are still sent to ES. Config is as follows:

    filebeat.inputs:
    - type: docker
      containers.ids:
      - '*'
      processors:
      - add_docker_metadata:
      - add_kubernetes_metadata:
          in_cluster: true
      - rename:
          fields:
          - from: "log_level"
            to: "level"
          - from: "log_tag"
            to: "tag"
          ignore_missing: true
      - drop_fields:
          fields: ["agent.ephemeral_id", "agent.hostname", "agent.id", "agent.type", "agent.version", "host.name", "ecs.version", "input.type"]
          ignore_missing: true
    output.elasticsearch:
      hosts: ["https://es-host:443"]
      protocol: "https"
      compression: 3
2

There is a global Processors section. Please use that. I tired it and it works
#================================Processors=====================================

#Configure processors to enhance or manipulate events generated by the beat.

processors:
  - drop_fields:
       fields: ["agent.ephemeral_id", "agent.hostname", "agent.id", "agent.type", "agent.version", "host.name", "ecs.version", "input.type"]
3

@kumarabhi thanks that worked!

4

Welcome. Please accept the answer as solution and close the topic.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.