# About Rollup

**URL:** <https://discuss.elastic.co/t/about-rollup/174931>\
**Category:** Elasticsearch\
**Created:** [April 2, 2019, 7:51am UTC](https://discuss.elastic.co/t/about-rollup/174931 "2019-04-02T07:51:11Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![iQIYI](https://avatars.discourse-cdn.com/v4/letter/i/bcef8e/32.png) [@iQIYI](https://discuss.elastic.co/u/iQIYI)\
**Post date:** [April 2, 2019, 7:51am UTC](https://discuss.elastic.co/t/about-rollup/174931/1 "2019-04-02T07:51:11Z")

</div>

Hello,  
I have some question about rollup API. Can anybody help me?

1. I already have about 100 daily indices. now I want to try rollup API, but I want to rollup from now on, can this be achieved?
2. As data growing, the rollup\_index shard size is getting bigger. But the Best Practices is set shard size between 20GB and 40GB, It's necessary to delete old data regularly?

60 nodes with ES 6.6.2

---

<div class="post-metadata">

**Author:** ![polyfractal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/polyfractal/32/48162_2.png) [@polyfractal](https://discuss.elastic.co/u/polyfractal)\
**Post date:** [April 5, 2019, 1:23pm UTC](https://discuss.elastic.co/t/about-rollup/174931/2 "2019-04-05T13:23:16Z")

</div>

Hm, I suppose that could be tricky to keep the rollup job from backfilling. You can specify a pattern (`logstash-*` for example), but that will also include all the existing indices that match.

Right now, the best way is probably with an alias that matches only the indices you care about. The difficulty is that you'll need to continually update the alias as new indices are added and old indices are removed.

We would like to add a bit more filtering capability to rollup jobs, but it's not possible right now.

> [@iQIYI](#):
>
> As data growing, the rollup\_index shard size is getting bigger. But the Best Practices is set shard size between 20GB and 40GB, It's necessary to delete old data regularly?

Agreed, this is a relatively important feature that's missing at the moment (issue here if you want to follow: [[Rollup] Managing index lifecycle · Issue #33065 · elastic/elasticsearch · GitHub](https://github.com/elastic/elasticsearch/issues/33065)). We plan to integrate Rollup with ILM soon, which will allow configuring custom rollover when the index hits a certain size, etc. Right now the only option is to let it grow, or start deleting data... neither of which are good solutions.

---

<div class="post-metadata">

**Author:** ![iQIYI](https://avatars.discourse-cdn.com/v4/letter/i/bcef8e/32.png) [@iQIYI](https://discuss.elastic.co/u/iQIYI)\
**Post date:** [April 8, 2019, 1:36am UTC](https://discuss.elastic.co/t/about-rollup/174931/3 "2019-04-08T01:36:44Z")

</div>

Thank you for your response 🙂

---

<div class="post-metadata">

**Author:** ![iQIYI](https://avatars.discourse-cdn.com/v4/letter/i/bcef8e/32.png) [@iQIYI](https://discuss.elastic.co/u/iQIYI)\
**Post date:** [April 8, 2019, 6:28am UTC](https://discuss.elastic.co/t/about-rollup/174931/4 "2019-04-08T06:28:04Z")

</div>

For the first question, I found a way to solve it. Before I create the rollup job, I closed the old indices. After this job finish indexing, I reopen those old indices, and the rollup index will not handle them

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 6, 2019, 6:28am UTC](https://discuss.elastic.co/t/about-rollup/174931/5 "2019-05-06T06:28:08Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
