# How to define a field when pushing the same document to multiple indexes?

**URL:** <https://discuss.elastic.co/t/how-to-define-a-field-when-pushing-the-same-document-to-multiple-indexes/165706>\
**Category:** Logstash\
**Created:** [January 25, 2019, 7:09am UTC](https://discuss.elastic.co/t/how-to-define-a-field-when-pushing-the-same-document-to-multiple-indexes/165706 "2019-01-25T07:09:39Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![jmkim](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jmkim/32/37997_2.png) [@jmkim](https://discuss.elastic.co/u/jmkim)\
**Post date:** [January 25, 2019, 7:09am UTC](https://discuss.elastic.co/t/how-to-define-a-field-when-pushing-the-same-document-to-multiple-indexes/165706/1 "2019-01-25T07:09:40Z")

</div>

Hello,

I am collecting logs and trying to output the logs to multiple indexes.  
Can I define a field for each index?

My Logstash output configuration is as follows.

> input: ip, name, url, msg

```
output {
      if "agg" in [tags] {
        elasticsearch {
          hosts => ["localhost:9200"]
          index => "agg_%{+YYYY.MM.dd}"
        }
      }
      if "err" in [tags] {
       elasticsearch {
          hosts => ["localhost:9200"]
          index => "err_%{+YYYY.MM.dd}"
        }
      }
    }

```

> current:  
> agg\_20190125: ip, name, url, msg  
> err\_20190125: ip, name, url, msg

> I want:  
> agg\_20190125: ip, name, url  
> err\_20190125: ip, msg

---

<div class="post-metadata">

**Author:** ![Shaoranlaos](https://avatars.discourse-cdn.com/v4/letter/s/c57346/32.png) [@Shaoranlaos](https://discuss.elastic.co/u/Shaoranlaos)\
**Post date:** [January 25, 2019, 9:32am UTC](https://discuss.elastic.co/t/how-to-define-a-field-when-pushing-the-same-document-to-multiple-indexes/165706/2 "2019-01-25T09:32:22Z")

</div>

I see three possibilities:

1. use Elasticsearch mapping parameter to exclude the fields from the documents  
for this the fields must be defined in the mapping of the index but with

2. Use logstash to clone the event (within a ruby filter) and than edit the events with logstash filters like you want them to be

3. Use a ingest pipeline an the index to remove the fields from the documents  
see [https://www.elastic.co/guide/en/elasticsearch/reference/current/pipeline.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/pipeline.html)  
you must then define the pipeline in the output definition of logstash with  
pipeline =\> "%{INGEST\_PIPELINE}"

---

<div class="post-metadata">

**Author:** ![jmkim](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jmkim/32/37997_2.png) [@jmkim](https://discuss.elastic.co/u/jmkim)\
**Post date:** [January 28, 2019, 2:41am UTC](https://discuss.elastic.co/t/how-to-define-a-field-when-pushing-the-same-document-to-multiple-indexes/165706/3 "2019-01-28T02:41:20Z")

</div>

Thank you, Shaoranlaos!  
I really appreciate your answers.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 25, 2019, 2:41am UTC](https://discuss.elastic.co/t/how-to-define-a-field-when-pushing-the-same-document-to-multiple-indexes/165706/4 "2019-02-25T02:41:24Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.