# Lifecycle Policy doesn't delete documents correctly

**URL:** <https://discuss.elastic.co/t/lifecycle-policy-doesnt-delete-documents-correctly/275105>\
**Category:** Elasticsearch\
**Tags:** ilm-index-lifecycle-management\
**Created:** [June 7, 2021, 6:53am UTC](https://discuss.elastic.co/t/lifecycle-policy-doesnt-delete-documents-correctly/275105 "2021-06-07T06:53:11Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![mar-ro](https://avatars.discourse-cdn.com/v4/letter/m/90ced4/32.png) [@mar-ro](https://discuss.elastic.co/u/mar-ro)\
**Post date:** [June 7, 2021, 6:53am UTC](https://discuss.elastic.co/t/lifecycle-policy-doesnt-delete-documents-correctly/275105/1 "2021-06-07T06:53:11Z")

</div>

Good morning!  
I set up a lifecycle policy as follows last week, in the hope that documents older than 3 days would be deleted but it hasn't worked, can you tell me why? My intention is to store the data for only that long, is there any way I could trigger the execution of that policy manually (or delete documents older than X days manually?). Thanks

```auto
PUT _ilm/policy/metricbeat
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {}
      },
      "delete": {
        "min_age": "3d",
        "actions": {
          "delete": {
            "delete_searchable_snapshot": true
          }
        }
      }
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [June 7, 2021, 7:03am UTC](https://discuss.elastic.co/t/lifecycle-policy-doesnt-delete-documents-correctly/275105/2 "2021-06-07T07:03:06Z")

</div>

What are the indices linked to this policy? How long time period does each index cover?

Note that ILM deletes complete indices older than 3 days and not individual documents.

---

<div class="post-metadata">

**Author:** ![mar-ro](https://avatars.discourse-cdn.com/v4/letter/m/90ced4/32.png) [@mar-ro](https://discuss.elastic.co/u/mar-ro)\
**Post date:** [June 7, 2021, 7:11am UTC](https://discuss.elastic.co/t/lifecycle-policy-doesnt-delete-documents-correctly/275105/3 "2021-06-07T07:11:05Z")

</div>

Thank you for your quick reply Christian.  
In the case of this policy, the only index you have linked is "metricbeat-7.12.1-2021.06.01-000001".  
Regarding how much time each index covers where could I check it, do you mean the "Maximum age" value of "Rollover" inside the hot phase?  
Thanks for the clarification that ILM removes the entire indexes and not the individual documents, anyway, it is still useful for my use case.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [June 7, 2021, 7:17am UTC](https://discuss.elastic.co/t/lifecycle-policy-doesnt-delete-documents-correctly/275105/4 "2021-06-07T07:17:48Z")

</div>

If I recall correctly I think timne start to cound once the index rolls over, which does not seem to have happened yet. How come your policy does not specify rollover parameters?

---

<div class="post-metadata">

**Author:** ![mar-ro](https://avatars.discourse-cdn.com/v4/letter/m/90ced4/32.png) [@mar-ro](https://discuss.elastic.co/u/mar-ro)\
**Post date:** [June 7, 2021, 7:26am UTC](https://discuss.elastic.co/t/lifecycle-policy-doesnt-delete-documents-correctly/275105/5 "2021-06-07T07:26:48Z")

</div>

I wasn't sure if it could be due to that and I tried to disable the "Rollover" option, however, in other indexes where it's enabled, the index hasn't been removed either.  
Taking into account what you said, that the time starts counting once the index is restarted, I'm going to test it by setting the "Maximum age" value to 2 minutes and the "Delete phase" to 5 minutes, so that every 2 minutes a new index is generated and every 5 minutes the previous one is deleted, should it work like that?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [June 7, 2021, 7:29am UTC](https://discuss.elastic.co/t/lifecycle-policy-doesnt-delete-documents-correctly/275105/6 "2021-06-07T07:29:39Z")

</div>

I believe ILM only checks every 10 minutes or so which can cause problems with unrealistic test settings.

If you generate hourly indices and set the delete phase to 2 hours I think the indices would get deleted once 2 hours have passed since they rolled over.

---

<div class="post-metadata">

**Author:** ![mar-ro](https://avatars.discourse-cdn.com/v4/letter/m/90ced4/32.png) [@mar-ro](https://discuss.elastic.co/u/mar-ro)\
**Post date:** [June 7, 2021, 7:31am UTC](https://discuss.elastic.co/t/lifecycle-policy-doesnt-delete-documents-correctly/275105/7 "2021-06-07T07:31:47Z")

</div>

Perfect, I will try those values, thank you very much!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2021, 7:32am UTC](https://discuss.elastic.co/t/lifecycle-policy-doesnt-delete-documents-correctly/275105/8 "2021-07-05T07:32:34Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.