# Reindex multiple matchind documents into a single document?

**URL:** https://discuss.elastic.co/t/reindex-multiple-matchind-documents-into-a-single-document/161077
**Category:** Logstash
**Created:** [December 17, 2018, 6:28am UTC](https://discuss.elastic.co/t/reindex-multiple-matchind-documents-into-a-single-document/161077 "2018-12-17T06:28:31Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![Sjaak01](https://avatars.discourse-cdn.com/v4/letter/s/73ab20/32.png) [@Sjaak01](https://discuss.elastic.co/u/Sjaak01)
#### Post date: [December 17, 2018, 6:28am UTC](https://discuss.elastic.co/t/reindex-multiple-matchind-documents-into-a-single-document/161077/1 "2018-12-17T06:28:31Z")

</div>

Hi,

I was wondering whether there is a practical way of querying existing documents in an index, find matching documents, and than write that that as a single document to a new index?

Example:  
I'm collecting network traffic and have the these fields:  
source  
destination  
port  
application  
in bytes  
out bytes

I have a lot of traffic so this quickly leads to lots of small documents and a heavy load on Elastic when performing big searches.

The idea is that after a while, it is not necessary to keep very detailed data anymore. E.g. after a month it is not necessary anymore to see the traffic at the second level, instead per day would be enough.

For that to work Logstash would need to search Elastic search for lets say a 24 hours period at a time, find documents where the source, destination, port and application match, sum up the in and out byes of all those fields and write the result to a new index as a single document.

The elastic filter appears it can do queries, but I'm not really sure it can handle all the necessary logic?

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [December 17, 2018, 6:33am UTC](https://discuss.elastic.co/t/reindex-multiple-matchind-documents-into-a-single-document/161077/2 "2018-12-17T06:33:34Z")

</div>

Another way to do this would be to use the [new rollup feature](https://www.elastic.co/blog/data-rollups-in-elasticsearch-you-know-for-saving-space) in Elasticsearch.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 14, 2019, 6:33am UTC](https://discuss.elastic.co/t/reindex-multiple-matchind-documents-into-a-single-document/161077/3 "2019-01-14T06:33:41Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
