# Rollup strategy in Elastic

**URL:** https://discuss.elastic.co/t/rollup-strategy-in-elastic/107438
**Category:** Elasticsearch
**Created:** [November 13, 2017, 4:46pm UTC](https://discuss.elastic.co/t/rollup-strategy-in-elastic/107438 "2017-11-13T16:46:42Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![John\_Doe1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/john_doe1/32/46751_2.png) [@John\_Doe1](https://discuss.elastic.co/u/John_Doe1)
#### Post date: [November 13, 2017, 4:46pm UTC](https://discuss.elastic.co/t/rollup-strategy-in-elastic/107438/1 "2017-11-13T16:46:42Z")

</div>

I am looking for a feasible way to rollup data I have stored in Elasticsearch. The records I have are time series based, and can be grouped by a timestamp, host, and path of a URL. What I had in mind was a cron job that looks at all the records 1 day old, not yet merged into a granularity. It would then bulk write the new merged records into the same index, and once completed run a delete by query where there is no granularity within the specified date range. I would eventually want to configure the cron job to run at the monthly/yearly granularity as well as single document granularity once it reaches a certain age.

What I am unsure about is the strategy needed to aggregate the data. For an input where there would be millions of records to aggregate, is this something I can handle with a single elastic query that fetches the aggregations, or will I have to use something such as Hadoop/MapReduce to read and aggregate the data? Would it be better to store granularities in separate indexes to make the rollup job easier?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 11, 2017, 4:46pm UTC](https://discuss.elastic.co/t/rollup-strategy-in-elastic/107438/2 "2017-12-11T16:46:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
