# Don't send tags to ElasticSearch

**URL:** <https://discuss.elastic.co/t/dont-send-tags-to-elasticsearch/130551>\
**Category:** Logstash\
**Created:** [May 4, 2018, 3:05am UTC](https://discuss.elastic.co/t/dont-send-tags-to-elasticsearch/130551 "2018-05-04T03:05:44Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![mrubin](https://avatars.discourse-cdn.com/v4/letter/m/e19adc/32.png) [@mrubin](https://discuss.elastic.co/u/mrubin)\
**Post date:** [May 4, 2018, 3:05am UTC](https://discuss.elastic.co/t/dont-send-tags-to-elasticsearch/130551/1 "2018-05-04T03:05:44Z")

</div>

I am processing kafka events and sending them to ES. I tag events so that I can tell which kafka topic they were from. I then use the tag in the output to direct events from different kafka topics to different ES indexes.

However, beyond their use in Logstash, I do not wish to send the tags to ElasticSearch and have to see them in Kibana. Is there any way to use the tags in logstash but prevent them from being sent to ES?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 4, 2018, 10:07am UTC](https://discuss.elastic.co/t/dont-send-tags-to-elasticsearch/130551/2 "2018-05-04T10:07:57Z")

</div>

You can have one final filter that removes the tags. There's no way to filter fields in the events in the elasticsearch output.

---

<div class="post-metadata">

**Author:** ![rcowart](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rcowart/32/88091_2.png) [@rcowart](https://discuss.elastic.co/u/rcowart)\
**Post date:** [May 4, 2018, 10:34am UTC](https://discuss.elastic.co/t/dont-send-tags-to-elasticsearch/130551/3 "2018-05-04T10:34:11Z")

</div>

Two options of the top of my head...

1. Use `@metadata.tags` instead of `tags`. @metadata fields are not forwarded by outputs, including the `elasticsearch` output. Your pipelines will need to use `add_field` instead of `add_tag` to populate this field. Or you can use `add_tag` and as the last step before your outputs rename `tags` to `@metadata.tags`. This will allow you to use `@metadata.tags` in your outputs to specify index names, but the field itself won't be sent.

2. Use an elasticsearch ingest pipeline to strip the tags field out of the incoming data before it is indexed.

---

<div class="post-metadata">

**Author:** ![mrubin](https://avatars.discourse-cdn.com/v4/letter/m/e19adc/32.png) [@mrubin](https://discuss.elastic.co/u/mrubin)\
**Post date:** [May 4, 2018, 6:57pm UTC](https://discuss.elastic.co/t/dont-send-tags-to-elasticsearch/130551/4 "2018-05-04T18:57:32Z")

</div>

@rcowart Using `@metadata` worked. Thank you both very much for the suggestions.

Regarding "elasticsearch ingest pipeline to strip the tags field out of the incoming data before it is indexed." are you referring to making a mapping or a dynamic mapping template for that field in that index such that `"index": false`? Would that be equivalent to stripping that field from the incoming data? Or does it mean that the field is still there but not "indexed" (meaning I can't search by it, but it is still there if I search by other fields?)?

Thank you.

---

<div class="post-metadata">

**Author:** ![rcowart](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rcowart/32/88091_2.png) [@rcowart](https://discuss.elastic.co/u/rcowart)\
**Post date:** [May 4, 2018, 9:41pm UTC](https://discuss.elastic.co/t/dont-send-tags-to-elasticsearch/130551/5 "2018-05-04T21:41:42Z")

</div>

I mean using the ingest node functionality of Elasticsearch. You can read more here...

[https://www.elastic.co/guide/en/elasticsearch/reference/master/ingest.html](https://www.elastic.co/guide/en/elasticsearch/reference/master/ingest.html)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 1, 2018, 9:41pm UTC](https://discuss.elastic.co/t/dont-send-tags-to-elasticsearch/130551/6 "2018-06-01T21:41:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
