# Merge two index to create third index using logstash

**URL:** <https://discuss.elastic.co/t/merge-two-index-to-create-third-index-using-logstash/249767>\
**Category:** Logstash\
**Created:** [September 24, 2020, 8:23am UTC](https://discuss.elastic.co/t/merge-two-index-to-create-third-index-using-logstash/249767 "2020-09-24T08:23:17Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Abhi6](https://avatars.discourse-cdn.com/v4/letter/a/c2a13f/32.png) [@Abhi6](https://discuss.elastic.co/u/Abhi6)\
**Post date:** [September 24, 2020, 8:23am UTC](https://discuss.elastic.co/t/merge-two-index-to-create-third-index-using-logstash/249767/1 "2020-09-24T08:23:17Z")

</div>

What I have done so far  
input {  
elasticsearch {  
hosts =\> "localost"  
index =\> "employees\_data,transaction\_data"

```
     query => '{ "query": { "match": { "code": 1} } }'
    scroll => "5m"
    docinfo => true
  }
}
output {

```

elasticsearch {  
hosts =\> ["localhost"]

```
index => "join1"
   }

```

}

It's giving me output like this  
{  
"took" : 4,  
"timed\_out" : false,  
"\_shards" : {  
"total" : 5,  
"successful" : 5,  
"skipped" : 0,  
"failed" : 0  
},  
"hits" : {  
"total" : 2,  
"max\_score" : 1.0,  
"hits" : [  
{  
"\_index" : "join1",  
"\_type" : "doc",  
"\_id" : "72gIv3QB\_L6Y9V8lNpCh",  
"\_score" : 1.0,  
"\_source" : {  
"@version" : "1",  
"@timestamp" : "2020-09-24T07:33:40.421Z",  
"payment" : 32080,  
"moth" : "june",  
"code" : 1  
}  
},  
{  
"\_index" : "join1",  
"\_type" : "doc",  
"\_id" : "8GgIv3QB\_L6Y9V8lN5AG",  
"\_score" : 1.0,  
"\_source" : {  
"city" : "indore",  
"@version" : "1",  
"@timestamp" : "2020-09-24T07:33:40.408Z",  
"name" : "Abhi",  
"salary" : 320800,  
"code" : 1  
}  
}  
]  
}  
}

How to get it in third index but one record based on code field

---

<div class="post-metadata">

**Author:** ![Hendrik\_Muhs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hendrik_muhs/32/25802_2.png) [@Hendrik\_Muhs](https://discuss.elastic.co/u/Hendrik_Muhs)\
**Post date:** [September 24, 2020, 2:56pm UTC](https://discuss.elastic.co/t/merge-two-index-to-create-third-index-using-logstash/249767/2 "2020-09-24T14:56:40Z")

</div>

I think logstash might not be the right tool, it's conceptually a mapper, what you need is a reducer. In practice you want to _group_ documents, in your case you want to group docs with the same code field. You need aggregations for that.

Long story short, please have a look at the very similar ask: [Merging documents based on matched fields values](https://discuss.elastic.co/t/merging-documents-based-on-matched-fields-values/249728/2)

As you explicitly say that you want an index as output, transform sounds like the right tool to me.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 22, 2020, 2:56pm UTC](https://discuss.elastic.co/t/merge-two-index-to-create-third-index-using-logstash/249767/3 "2020-10-22T14:56:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.