# Unable to Drop desired fields using drop\_fields processors in filebeat

**URL:** <https://discuss.elastic.co/t/unable-to-drop-desired-fields-using-drop-fields-processors-in-filebeat/264654>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [February 18, 2021, 6:53am UTC](https://discuss.elastic.co/t/unable-to-drop-desired-fields-using-drop-fields-processors-in-filebeat/264654 "2021-02-18T06:53:33Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![mauryaravi](https://avatars.discourse-cdn.com/v4/letter/m/aca169/32.png) [@mauryaravi](https://discuss.elastic.co/u/mauryaravi)\
**Post date:** [February 18, 2021, 6:53am UTC](https://discuss.elastic.co/t/unable-to-drop-desired-fields-using-drop-fields-processors-in-filebeat/264654/1 "2021-02-18T06:53:33Z")

</div>

```auto
I am sending log data to elastic search directly using filebeat 7.10.
I want to drop some fields of doc using drop_fields processors before sending to ES. Filebeat is dropping some fields placed in drop_fields conf but not all. Filebeat throwing following error:

DEBUG	[processors]	processing/processors.go:128	Fail to apply processor client{drop_fields={"Fields":["agent","host","log","input","ecs"],"IgnoreMissing":false}}: failed to drop field [agent]: key not found; failed to drop field [host]: key not found; failed to drop field [ecs]: key not found

```

```auto
my filebeat.yml-------------
filebeat.inputs:
- type: log
  processors:
    - drop_fields:
        fields: ["agent", "host", "log", "input", "ecs"]

setup.template.settings:
  index.number_of_shards: 1

setup.template:
  name: "test"
  enabled: true
  pattern: "test-*"

setup.ilm:
  enabled: false

setup.dashboards.enabled: true
setup.dashboards.index: "test-*"

setup.kibana:
host: "192.111.1.1:5601"

output.elasticsearch:
  hosts: ["192.111.1.1:9200"]

FileBeat logs *********************
DEBUG	[processors]	processing/processors.go:128	Fail to apply processor client{drop_fields={"Fields":["agent","host","log","input","ecs"],"IgnoreMissing":false}}: failed to drop field [agent]: key not found; failed to drop field [host]: key not found; failed to drop field [ecs]: key not found
2021-02-18T06:28:40.804Z	DEBUG	[processors]	processing/processors.go:203	Publish event: {
  "@timestamp": "2021-02-18T06:28:40.804Z",
  "@metadata": {
    "beat": "filebeat",
    "type": "_doc",
    "version": "7.10.0"
  },
  "message": "qqqqqqqqqqqqqqqqqqq",
  "agent": {
    "hostname": "usps-backend1",
    "ephemeral_id": "f9239289-81fe-4fa9-ac46-8ed81e0edcc0",
    "id": "e35682f8-8fa6-47e0-b4fa-af78c2170d26",
    "name": "usps-backend1",
    "type": "filebeat",
    "version": "7.10.0"
  },
  "ecs": {
    "version": "1.6.0"
  },
  "host": {
    "name": "usps-backend1",
    "hostname": "usps-backend1",
    "architecture": "x86_64",
    "os": {
      "codename": "stretch",
      "platform": "debian",
      "version": "9 (stretch)",
      "family": "debian",
      "name": "Debian GNU/Linux",
      "kernel": "4.9.0-11-amd64"
    },
    "id": "aca0b202602c41c4aed5fe533c64ff85",
    "containerized": false,
    "ip": [
      "172.25.1.66",
      "fe80::f816:3eff:fe2a:5757"
    ],
    "mac": [
      "fa:16:3e:2a:57:57"
    ]
  }
}

```

---

<div class="post-metadata">

**Author:** ![mauryaravi](https://avatars.discourse-cdn.com/v4/letter/m/aca169/32.png) [@mauryaravi](https://discuss.elastic.co/u/mauryaravi)\
**Post date:** [February 18, 2021, 10:25am UTC](https://discuss.elastic.co/t/unable-to-drop-desired-fields-using-drop-fields-processors-in-filebeat/264654/2 "2021-02-18T10:25:26Z")

</div>

````auto
Problem is solved now.
I was configuring processor at rwong location ki filebeat.yml.
```
````

---

<div class="post-metadata">

**Author:** ![lachezar.uzunov](https://avatars.discourse-cdn.com/v4/letter/l/ecae2f/32.png) [@lachezar.uzunov](https://discuss.elastic.co/u/lachezar.uzunov)\
**Post date:** [February 26, 2021, 12:15pm UTC](https://discuss.elastic.co/t/unable-to-drop-desired-fields-using-drop-fields-processors-in-filebeat/264654/3 "2021-02-26T12:15:17Z")

</div>

Can you post the config changes?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 26, 2021, 2:15pm UTC](https://discuss.elastic.co/t/unable-to-drop-desired-fields-using-drop-fields-processors-in-filebeat/264654/4 "2021-03-26T14:15:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.