Filebeat didn't drop some of the fields like agent.*, ecs.* etc

Elastic Stack Beats
,
1

Hi,

I am using filebeat with a docker processor. Filebeat generate some fields like agent, ecs etc. I am trying to remove these fields using drop_fields processor.

filebeat.inputs:
- type: docker
  containers.ids: 
    - '*'
  processors:
  - add_docker_metadata: ~
  - drop_fields:
      fields: ["container.image","container.labels", "agent.ephemeral_id", "agent.version", 
      "ecs.version"]
      ignore_missing: true
  setup.dashboards.enabled: true

First two fields i.e container.image, container.labels are emoved but rest are not.

1 Like
2

The fields agent.ephemeral_id, agent.version and ecs.version are added later in the processing pipeline. In order to remove those you need to add a drop_fields processor to the global level.

filebeat.inputs:
- type: docker
  containers.ids: 
    - '*'
  processors:
  - add_docker_metadata: ~
  - drop_fields:
      fields: ["container.image","container.labels"]
      ignore_missing: true

processors:
- drop_fields:
    fields: ["agent.ephemeral_id", "agent.version", "ecs.version"]
    ignore_missing: true

setup.dashboards.enabled: true
2 Likes
3

Thank you so much. Its working :slightly_smiling_face:

1 Like
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.