# Misunderstood ILM **URL:** \ **Category:** Elasticsearch\ **Tags:** ilm-index-lifecycle-management\ **Created:** [September 1, 2020, 7:39am UTC](https://discuss.elastic.co/t/misunderstood-ilm/247057 "2020-09-01T07:39:50Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![headtea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/headtea/32/24271_2.png) [@headtea](https://discuss.elastic.co/u/headtea)\ **Post date:** [September 1, 2020, 7:39am UTC](https://discuss.elastic.co/t/misunderstood-ilm/247057/1 "2020-09-01T07:39:50Z")
I was looking at a solution to delete old logs and I was suggested to set up ILM. I only now realize that ILM rolls over the entire index rather than old entries. If I understand correctly if I set up ILM to 3 months and then move to deletion, after 3 months it would delete the entire index. Is there a way to set it so after 3 months, **only entires** that are older than 3 months will be deleted? Is it possible to do so for storange as well (once reaches 250GB delete old entires)? Thanks ahead! ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 1, 2020, 7:41am UTC](https://discuss.elastic.co/t/misunderstood-ilm/247057/2 "2020-09-01T07:41:28Z")
You'd need to create a custom process to handle that with a delete-by-query. It's highly inefficient though, and you'd be better off using ILM and deleting entire indices instead. ---
**Author:** ![headtea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/headtea/32/24271_2.png) [@headtea](https://discuss.elastic.co/u/headtea)\ **Post date:** [September 1, 2020, 7:51am UTC](https://discuss.elastic.co/t/misunderstood-ilm/247057/3 "2020-09-01T07:51:12Z")
Thanks for the response, The problem with that is that this would delete indexes with very recent data. I assume there's a best practice I'm missing here. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 1, 2020, 7:53am UTC](https://discuss.elastic.co/t/misunderstood-ilm/247057/4 "2020-09-01T07:53:30Z")
Not sure I follow why that would be the case? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [September 1, 2020, 7:54am UTC](https://discuss.elastic.co/t/misunderstood-ilm/247057/5 "2020-09-01T07:54:49Z")
Best practice is to use time-based indices where data gets assigned to indices based on when they occur. The oldest indices therefore hold the oldest data and indices are deleted oldest first when all data in them has exceeded the retention period. ---
**Author:** ![headtea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/headtea/32/24271_2.png) [@headtea](https://discuss.elastic.co/u/headtea)\ **Post date:** [September 1, 2020, 8:00am UTC](https://discuss.elastic.co/t/misunderstood-ilm/247057/6 "2020-09-01T08:00:18Z")
Thanks for the response! May I ask for example how to set it so that logstash would create a new indice each month? Right now this is my output in logstash: ``` index => "%{[@metadata][beat]}-%{[@metadata][version]}" ``` ---
**Author:** ![headtea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/headtea/32/24271_2.png) [@headtea](https://discuss.elastic.co/u/headtea)\ **Post date:** [September 1, 2020, 8:02am UTC](https://discuss.elastic.co/t/misunderstood-ilm/247057/7 "2020-09-01T08:02:53Z")
Because as of now, I'm not creating time based indices. all the output goes to one single indice. This means that if it gets 3 months old it would be deleted, even though it recieved a document 5 minutes ago. Sorry if I'm not clear, this is pretty new to me. Please tell me if I could phrase it better. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 1, 2020, 8:23am UTC](https://discuss.elastic.co/t/misunderstood-ilm/247057/8 "2020-09-01T08:23:48Z")
No worries, thanks for explaining your current approach! Have a read of [this page](https://www.elastic.co/guide/en/elasticsearch/reference/current/example-using-index-lifecycle-policy.html) of the docs, it gives a practical example of how this would work, the overall approach is how it'd work with Logstash as well. ---
**Author:** ![headtea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/headtea/32/24271_2.png) [@headtea](https://discuss.elastic.co/u/headtea)\ **Post date:** [September 1, 2020, 8:41am UTC](https://discuss.elastic.co/t/misunderstood-ilm/247057/9 "2020-09-01T08:41:12Z")
Thanks for the response, I couldn't find logstash output reference in that guide. If in logstash I do: ``` index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}" ``` and in ILM simply do: Hot for 3 months -\> delete. Would this remove day worth of data everyday once it reaches 3 months? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 1, 2020, 8:49pm UTC](https://discuss.elastic.co/t/misunderstood-ilm/247057/10 "2020-09-01T20:49:48Z")
Ok try [https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-ilm](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-ilm) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 29, 2020, 8:49pm UTC](https://discuss.elastic.co/t/misunderstood-ilm/247057/11 "2020-09-29T20:49:48Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.