# Filebeat Drop field doesnt work for json

**URL:** <https://discuss.elastic.co/t/filebeat-drop-field-doesnt-work-for-json/248432>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [September 13, 2020, 6:53am UTC](https://discuss.elastic.co/t/filebeat-drop-field-doesnt-work-for-json/248432 "2020-09-13T06:53:41Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![niv](https://avatars.discourse-cdn.com/v4/letter/n/eb9ed0/32.png) [@niv](https://discuss.elastic.co/u/niv)\
**Post date:** [September 13, 2020, 6:53am UTC](https://discuss.elastic.co/t/filebeat-drop-field-doesnt-work-for-json/248432/1 "2020-09-13T06:53:41Z")

</div>

```````````````````````````````auto
filebeat.inputs:
      - type: log
        enabled: true
        paths:
          - "*.json"
processors:
- drop_fields:
    fields: ["value"]
``````````````````````````````

That's my filebeat yml file. the drop fields doesn't work, any suggestions?
```````````````````````````````

---

<div class="post-metadata">

**Author:** ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)\
**Post date:** [September 14, 2020, 2:37pm UTC](https://discuss.elastic.co/t/filebeat-drop-field-doesnt-work-for-json/248432/2 "2020-09-14T14:37:04Z")

</div>

Hi @niv, welcome to the Elastic Community Forums!

Can you post a sample event that's being published by Filebeat with this configuration? To do so it might be convenient to temporarily change your Filebeat `output` to [`console`](https://www.elastic.co/guide/en/beats/filebeat/current/console-output.html).

Also, just guessing based on your your log file `paths` setting, I wonder if you need to use the [`json`](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-log.html#filebeat-input-log-config-json) setting under your `log` input configuration?

Shaunak

---

<div class="post-metadata">

**Author:** ![niv](https://avatars.discourse-cdn.com/v4/letter/n/eb9ed0/32.png) [@niv](https://discuss.elastic.co/u/niv)\
**Post date:** [September 15, 2020, 5:23pm UTC](https://discuss.elastic.co/t/filebeat-drop-field-doesnt-work-for-json/248432/3 "2020-09-15T17:23:50Z")

</div>

heres my JSON :

```auto
[
    {
        "value": "dropfields_3",
        "PSN": "dropfields_3",
        "Id": "dropfields_3",
        "PSShowComputerName": "dropfields_3"
    }
]

```

```auto
filebeat.inputs:
      - type: log
        enabled: true
        paths:
          - "*.json"
        scan_frequency: 1s
        tail_files: false
        multiline.pattern: '{'
        multiline.negate: true
        multiline.match: after
        clean_removed: true
        ignore_older: 336h
        close_inactive: 24h

processors:
      - drop_fields:
        fields: ["value"]

```

---

<div class="post-metadata">

**Author:** ![niv](https://avatars.discourse-cdn.com/v4/letter/n/eb9ed0/32.png) [@niv](https://discuss.elastic.co/u/niv)\
**Post date:** [September 15, 2020, 6:19pm UTC](https://discuss.elastic.co/t/filebeat-drop-field-doesnt-work-for-json/248432/4 "2020-09-15T18:19:59Z")

</div>

HERE"S THE OUTPUT ON CONSOLE :

```auto
{
  "@timestamp": "2020-09-15T18:15:45.372Z",
  "@metadata": {
    "beat": "filebeat",
    "type": "_doc",
    "version": "7.3.0"
  },
  "log": {
    "offset": 0,
    "file": {
      "path": "sample.json"
    }
  },
  "message": "[",
  "input": {
    "type": "log"
  },
  "ecs": {
    "version": "1.0.1"
  },
  "host": {
    "name": "xxxxxxxxxxxxxxxxxxxxxxxxx"
  },
  "agent": {
    "ephemeral_id": "xxxxxxxxxxxxxxxxxxxx",
    "hostname": "xxxxxxxxxxxxx",
    "id": "xxxxxxxxxxxxxxxxxxxxx",
    "version": "7.3.0",
    "type": "filebeat"
  }
}
{
  "@timestamp": "2020-09-15T18:15:45.372Z",
  "@metadata": {
    "beat": "filebeat",
    "type": "_doc",
    "version": "7.3.0"
  },
  "log": {
    "flags": [
      "multiline"
    ],
    "offset": 3,
    "file": {
      "path": "sample.json"
    }
  },
  "message": " {\n \"value\": \"dropfields_3\",\n \"PSN\": \"dropfields_3\",\n \"Id\": \"dropfields_3\",\n \"PSShowComputerName\": \"drop
fields_3\"\n }",
  "input": {
    "type": "log"
  },
  "ecs": {
    "version": "x.x.x"
  },
  "host": {
    "name": "xxxxxxxxxxxxxxxxxx"
  },
  "agent": {
    "version": "7.3.0",
    "type": "filebeat",
    "ephemeral_id": "xxxxxxxxxxxxxxxxxx",
    "hostname": "xxxxxxxxxxxxxxxxxxxx",
    "id": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
  }
}

```

I also dont want the extra fields to appear.

---

<div class="post-metadata">

**Author:** ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)\
**Post date:** [September 15, 2020, 7:19pm UTC](https://discuss.elastic.co/t/filebeat-drop-field-doesnt-work-for-json/248432/5 "2020-09-15T19:19:10Z")

</div>

Hi, please edit your posts and enclose your code/config/log snippets in ``` delimiters so we can read them more easily with the indentation in place.

Thanks,

Shaunak

---

<div class="post-metadata">

**Author:** ![niv](https://avatars.discourse-cdn.com/v4/letter/n/eb9ed0/32.png) [@niv](https://discuss.elastic.co/u/niv)\
**Post date:** [September 15, 2020, 7:33pm UTC](https://discuss.elastic.co/t/filebeat-drop-field-doesnt-work-for-json/248432/6 "2020-09-15T19:33:39Z")

</div>

Done, I also need help to remove the unwanted fields that filebeat adds like @version, @metadata, agent, etc. I tried "fields\_under\_root : false" but that didn't work .

---

<div class="post-metadata">

**Author:** ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)\
**Post date:** [September 16, 2020, 10:10am UTC](https://discuss.elastic.co/t/filebeat-drop-field-doesnt-work-for-json/248432/7 "2020-09-16T10:10:02Z")

</div>

Is your entire log file one JSON array containing multiple JSON objects (one object per log entry) or is it one JSON array containing one JSON object (one array per log entry)?

Shaunak

---

<div class="post-metadata">

**Author:** ![niv](https://avatars.discourse-cdn.com/v4/letter/n/eb9ed0/32.png) [@niv](https://discuss.elastic.co/u/niv)\
**Post date:** [September 16, 2020, 5:48pm UTC](https://discuss.elastic.co/t/filebeat-drop-field-doesnt-work-for-json/248432/8 "2020-09-16T17:48:49Z")

</div>

that was just a sample file I was using to test. Typically it would look like this.

```auto

[
    {
        "value": "dropfields_3",
        "PSN": "dropfields_3",
        "Id": "dropfields_3",
        "PSShowComputerName": "dropfields_3"
    },
	    {
        "value": "dropfields_1",
        "PSN": "dropfields_1",
        "Id": "dropfields_1",
        "PSShowComputerName": "dropfields_1"
    },
	    {
        "value": "dropfields_2",
        "PSN": "dropfields_2",
        "Id": "dropfields_2",
        "PSShowComputerName": "dropfields_2"
    }
]

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 14, 2020, 7:48pm UTC](https://discuss.elastic.co/t/filebeat-drop-field-doesnt-work-for-json/248432/9 "2020-10-14T19:48:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.