# \[ILM Policy\] Best compromise between size and max age

**URL:** <https://discuss.elastic.co/t/ilm-policy-best-compromise-between-size-and-max-age/223299>\
**Category:** Elasticsearch\
**Created:** [March 12, 2020, 9:35am UTC](https://discuss.elastic.co/t/ilm-policy-best-compromise-between-size-and-max-age/223299 "2020-03-12T09:35:24Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Travis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/travis/32/54079_2.png) [@Travis](https://discuss.elastic.co/u/Travis)\
**Post date:** [March 12, 2020, 9:35am UTC](https://discuss.elastic.co/t/ilm-policy-best-compromise-between-size-and-max-age/223299/1 "2020-03-12T09:35:24Z")

</div>

Hello Elasticians !

While testing ILM policy feature, I've just figured that the `delete` phase is relative to the rollover time, not the index creation time [(source)](https://www.elastic.co/guide/en/elasticsearch/reference/current/getting-started-index-lifecycle-management.html)

Let's say I have the following ILM policy :

> {  
> "ims\_customer\_policy" : {  
> "version" : 1,  
> "modified\_date" : "2019-10-02T09:59:43.574Z",  
> "policy" : {  
> "phases" : {  
> "hot" : {  
> "min\_age" : "0ms",  
> "actions" : {  
> "rollover" : {  
> "max\_size" : "30gb"  
> }  
> }  
> },  
> "delete" : {  
> "min\_age" : "365d",  
> "actions" : {  
> "delete" : { }  
> }  
> }  
> }  
> }  
> }  
> }

This way, all index have a regular size (which is a good practice I think) but I can't be sure that logs will be conserved one year because an index could take one hour, or three days or whatever to reach the size of 30 GB right ? Some could be conserved some month while other could be conserved more than one year

My only option would be to add `max age : 1` to my hot phase, but index size will not be regular (i.e more logs on week and less in weekends)

So, what is your best strategy to have a good compromise between retention time and index size ?

Thanks for your feedback !

---

<div class="post-metadata">

**Author:** ![Travis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/travis/32/54079_2.png) [@Travis](https://discuss.elastic.co/u/Travis)\
**Post date:** [March 12, 2020, 10:27pm UTC](https://discuss.elastic.co/t/ilm-policy-best-compromise-between-size-and-max-age/223299/2 "2020-03-12T22:27:22Z")

</div>

Anyone ? @Christian_Dahlqvist @DavidTurner would you have some feedback on this ? 😉

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [March 13, 2020, 2:21am UTC](https://discuss.elastic.co/t/ilm-policy-best-compromise-between-size-and-max-age/223299/3 "2020-03-13T02:21:34Z")

</div>

I would recommend setting a max age for the hot phase of perhaps a week as this seems reasonable given your retention period. If this results in some indices that are smaller than the size target that should not be a problem.

Then adjust you retention settings so you always keep at least a years worth of data. If the oldest index covers a full week this means you at times will hold a year plus a week which in my experience generally is fine.

---

<div class="post-metadata">

**Author:** ![Travis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/travis/32/54079_2.png) [@Travis](https://discuss.elastic.co/u/Travis)\
**Post date:** [March 13, 2020, 1:43pm UTC](https://discuss.elastic.co/t/ilm-policy-best-compromise-between-size-and-max-age/223299/4 "2020-03-13T13:43:40Z")

</div>

Hello @Christian_Dahlqvist. Mmh. I just have hot nodes as I don't have enough resources to have warm and cold nodes 😑

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [March 14, 2020, 8:44am UTC](https://discuss.elastic.co/t/ilm-policy-best-compromise-between-size-and-max-age/223299/5 "2020-03-14T08:44:49Z")

</div>

That does not matter. Moving shards between zones is one aspect of the index lifecycle, but there are other steps that apply even if you only have a single zone.

---

<div class="post-metadata">

**Author:** ![Travis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/travis/32/54079_2.png) [@Travis](https://discuss.elastic.co/u/Travis)\
**Post date:** [March 14, 2020, 1:40pm UTC](https://discuss.elastic.co/t/ilm-policy-best-compromise-between-size-and-max-age/223299/6 "2020-03-14T13:40:54Z")

</div>

Ok thank you @Christian_Dahlqvist. I'm gonna try this and see what is going on !

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 11, 2020, 1:43pm UTC](https://discuss.elastic.co/t/ilm-policy-best-compromise-between-size-and-max-age/223299/7 "2020-04-11T13:43:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.