# Ignore specific fields?

**URL:** <https://discuss.elastic.co/t/ignore-specific-fields/89182>\
**Category:** Logstash\
**Created:** [June 13, 2017, 11:50am UTC](https://discuss.elastic.co/t/ignore-specific-fields/89182 "2017-06-13T11:50:10Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kissaki](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kissaki/32/12437_2.png) [@Kissaki](https://discuss.elastic.co/u/Kissaki)\
**Post date:** [June 13, 2017, 11:50am UTC](https://discuss.elastic.co/t/ignore-specific-fields/89182/1 "2017-06-13T11:50:10Z")

</div>

Hey,

I'm using logstash to move already parsed messages in a json format from redis to elasticsearch. To determine the index a document is indexed to, I have a @index field in that message and a @type field for the type of the document.

In logstash, I can set the index in the output:

```
output {
  elasticsearch {
    hosts => ["http://127.0.0.1:9200"]
    index => "%{@index}"
  }
}

```

And the type in the filter via mutate:

```
filter{
    mutate {
        replace => { "type" => "%{@type}" }
    }
}

```

These fields however end up in elasticsearch, which I don't want to happen. I can remove them via remove\_field in the filter, but that will mess up the index in the output section. Is there any way to tell logstash to ignore those fields for the output only?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [June 13, 2017, 11:57am UTC](https://discuss.elastic.co/t/ignore-specific-fields/89182/2 "2017-06-13T11:57:48Z")

</div>

You can copy these fields to the [metadata](https://www.elastic.co/guide/en/logstash/5.4/event-dependent-configuration.html#metadata) and then delete them from the actual event. You can then use the metadata fields in the output without having them in the actual event being written to Elasticsearch.

---

<div class="post-metadata">

**Author:** ![Kissaki](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kissaki/32/12437_2.png) [@Kissaki](https://discuss.elastic.co/u/Kissaki)\
**Post date:** [June 20, 2017, 10:44am UTC](https://discuss.elastic.co/t/ignore-specific-fields/89182/3 "2017-06-20T10:44:55Z")

</div>

Metadata works now at least for assigning the index. Now that the index works (assigning a deleted field to an index, causes errors and un-deletable indexes in cerebro), it seems the fields are still transported to elasticsearch.

In my logstash config I have:

```
mutate { add_field => { "[@metadata][index]" => "%{@index}" } }
mutate {
    replace => { "type" => "%{@type}" }
}
mutate {
    remove_field => ["%{@index}","%{@version}","%{@type}"]
}

```

When I transmit some data and take a look into kibana, I still have the fields I tried to filter out:

```
t @index testindex-103
@timestamp June 20th 2017, 12:38:07.860
t @type filler
t @version 1

```

What am I doing wrong here?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 20, 2017, 10:46am UTC](https://discuss.elastic.co/t/ignore-specific-fields/89182/4 "2017-06-20T10:46:05Z")

</div>

`remove_field` should list the _names_ of the fields and not their contents, so do this instead:

```
remove_field => ["@index","@version","@type"]
```

---

<div class="post-metadata">

**Author:** ![Kissaki](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kissaki/32/12437_2.png) [@Kissaki](https://discuss.elastic.co/u/Kissaki)\
**Post date:** [June 20, 2017, 11:02am UTC](https://discuss.elastic.co/t/ignore-specific-fields/89182/5 "2017-06-20T11:02:45Z")

</div>

That was the problem, thanks for the quick reply. It does work now 😄

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 18, 2017, 11:02am UTC](https://discuss.elastic.co/t/ignore-specific-fields/89182/6 "2017-07-18T11:02:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.