# How to remove fields in logstash/es

**URL:** <https://discuss.elastic.co/t/how-to-remove-fields-in-logstash-es/77039>\
**Category:** Logstash\
**Created:** [March 1, 2017, 6:37pm UTC](https://discuss.elastic.co/t/how-to-remove-fields-in-logstash-es/77039 "2017-03-01T18:37:35Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![bob-bza](https://avatars.discourse-cdn.com/v4/letter/b/9f8e36/32.png) [@bob-bza](https://discuss.elastic.co/u/bob-bza)\
**Post date:** [March 1, 2017, 6:37pm UTC](https://discuss.elastic.co/t/how-to-remove-fields-in-logstash-es/77039/1 "2017-03-01T18:37:35Z")

</div>

This was my index looks like in kibana.  
I don't want so many fields below, so how can i delete/avoid those from my index.  
Ex.: id, \_score, version, beat info.

```auto
{
  "_index": "test",
  "_type": "testing",
  "_id": "AVqK6n0wvKz7",
  "_score": null,
  "_source": {
    "@timestamp": "2017-03-01T17:27:48.830Z",
    "offset": 207,
    "@version": "1",
    "input_type": "log",
    "beat": {
      "hostname": "ip",
      "name": "ipl",
      "version": "5.0.2"
    },
    "host": "ipl",
    "source": "/var/log/ambari.log",
    "message": "ERROR [main] DBAccessorImpl:109 - Error while creating database accessororg.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.",
    "type": "error-log",
    "tags": [
      "ambari",
      "beats_input_codec_plain_applied",
      "_grokparsefailure"
    ]
  },
  "fields": {
    "@timestamp": [
      1488389268830
    ]
  },
  "sort": [
    1488389268830
  ]
}

```

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [March 1, 2017, 6:56pm UTC](https://discuss.elastic.co/t/how-to-remove-fields-in-logstash-es/77039/2 "2017-03-01T18:56:21Z")

</div>

**Note:** Please encapsulate your code/console pasts within triple-backticks: ```. I've done this for you.

Removing them in ES is a full delete-\>reindex operation. Very doable, but perhaps not easy.

Really, the only things that actually come in your document are the things in `_source`. You can't eliminate the `_index`, `_type`, `_id`, and `_source` fields as they are ES metadata. `_score` is generated at search time, so it's not actually in your document.

`@timestamp` is necessary, and `beat` (and its sub keys) are used as identifiers, but can possibly be deleted. The others are perhaps up to you.

It's pretty easy to remove fields in Logstash:

```auto
filter {
  mutate { remove_field => ["field1", "field2", "field3", ... "fieldN"] }
}

```

---

<div class="post-metadata">

**Author:** ![bob-bza](https://avatars.discourse-cdn.com/v4/letter/b/9f8e36/32.png) [@bob-bza](https://discuss.elastic.co/u/bob-bza)\
**Post date:** [March 1, 2017, 9:01pm UTC](https://discuss.elastic.co/t/how-to-remove-fields-in-logstash-es/77039/3 "2017-03-01T21:01:09Z")

</div>

> [@theuntergeek](#):
>
> filter {  
> mutate { remove\_field =\> ["field1", "field2", "field3", ... "fieldN"] }  
> }

i added in my filters, i tried both cases it doesn't work for me.  
mutate { remove\_field =\> ["version"] }  
mutate { remove\_field =\> ["@version"] }

am i missing anything ?

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [March 1, 2017, 9:14pm UTC](https://discuss.elastic.co/t/how-to-remove-fields-in-logstash-es/77039/4 "2017-03-01T21:14:26Z")

</div>

`@version` probably can't be removed as it is a protected field in Logstash.

---

<div class="post-metadata">

**Author:** ![bob-bza](https://avatars.discourse-cdn.com/v4/letter/b/9f8e36/32.png) [@bob-bza](https://discuss.elastic.co/u/bob-bza)\
**Post date:** [March 1, 2017, 9:34pm UTC](https://discuss.elastic.co/t/how-to-remove-fields-in-logstash-es/77039/5 "2017-03-01T21:34:18Z")

</div>

> [@bob-bza](#):
>
> \_id": "AVqK6n0wvKz7",  
> "\_score": null,

"\_index": "test",  
"\_type": "testing",  
"\_id": "AVqK6n0wvKz7",  
"\_score": null,

i want to remove these fields , how can i remove those?

mutate { remove\_field =\> ["\_id", "input\_type", "[beat][version]", "[beat][name]" ] }

was not able to remove "\_id" rest everything i can remove

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [March 1, 2017, 9:37pm UTC](https://discuss.elastic.co/t/how-to-remove-fields-in-logstash-es/77039/6 "2017-03-01T21:37:54Z")

</div>

Please re-read what I said above:

> [@theuntergeek](#):
>
> Really, the only things that actually come in your document are the things in \_source. You can't eliminate the \_index, \_type, \_id, and \_source fields as they are ES metadata. \_score is generated at search time, so it's not actually in your document.

The fields inside of `_source` are the ones that come from Logstash. You cannot remove the fields outside of `_source`.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 29, 2017, 9:38pm UTC](https://discuss.elastic.co/t/how-to-remove-fields-in-logstash-es/77039/7 "2017-03-29T21:38:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.