# Index life cycle policy to delete indices

**URL:** <https://discuss.elastic.co/t/index-life-cycle-policy-to-delete-indices/300649>\
**Category:** Elasticsearch\
**Tags:** ilm-index-lifecycle-management\
**Created:** [March 25, 2022, 12:09am UTC](https://discuss.elastic.co/t/index-life-cycle-policy-to-delete-indices/300649 "2022-03-25T00:09:36Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![tkotthakota](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tkotthakota/32/81695_2.png) [@tkotthakota](https://discuss.elastic.co/u/tkotthakota)\
**Post date:** [March 25, 2022, 12:09am UTC](https://discuss.elastic.co/t/index-life-cycle-policy-to-delete-indices/300649/1 "2022-03-25T00:09:36Z")

</div>

Hi, I created below policy to delete old indices. I notice that old indices are getting deleted but slowly . Meaning the count went down from 695 to 573 in 1 hour. Is it normal?

```auto
PUT _ilm/policy/QA_only_cleanup_old_indices
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "set_priority": {
            "priority": 100
          }
        }
      },
      "delete": {
        "min_age": "3d",
        "actions": {
          "delete": {
            "delete_searchable_snapshot": true
          }
        }
      }
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [March 28, 2022, 12:13am UTC](https://discuss.elastic.co/t/index-life-cycle-policy-to-delete-indices/300649/2 "2022-03-28T00:13:56Z")

</div>

Welcome to our community! 😃

I know ILM runs every 10 minutes by default. I am not sure if it has internal limits around how many actions it'll take as a result of that run.

---

<div class="post-metadata">

**Author:** ![tkotthakota](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tkotthakota/32/81695_2.png) [@tkotthakota](https://discuss.elastic.co/u/tkotthakota)\
**Post date:** [March 28, 2022, 3:41pm UTC](https://discuss.elastic.co/t/index-life-cycle-policy-to-delete-indices/300649/3 "2022-03-28T15:41:40Z")

</div>

One thing I noticed that even though indices are getting purged I still see indices older than 3 days.  
We have date as creation.date.string.  
Here is the query I am using to get all indices:  
`GET _cat/indices/catalog_1*?v&h=h,s,i,id,p,r,dc,dd,ss,creation.date.string&s=creation.date:desc`  
Wondering what changes needed to the above policy I posted few days back to make it work.

Example indices which are older than 3 days:

```auto
green open catalog_1_7f64fbab-6797-4047-ac79-f50a5945ba9b_test_upaj73nx_202203080206_40dbefde 6Ano0zH-QsW7PqjEyob4oA 1 2 1171 0 1.3mb 2022-03-08T02:07:32.222Z
green open catalog_1_7f64fbab-6797-4047-ac79-f50a5945ba9b_test_sha2nvq4_202203072005_4a4bc593 oh7movLcTwyWMpgoMQNFgA 1 2 1171 0 1.3mb 2022-03-07T20:07:26.270Z

```

Here is the explain for one of the indices:

```auto
{
  "indices" : {
    "catalog_1_7f64fbab-6797-4047-ac79-f50a5945ba9b_test_sha2nvq4_202203072005_4a4bc593" : {
      "index" : "catalog_1_7f64fbab-6797-4047-ac79-f50a5945ba9b_test_sha2nvq4_202203072005_4a4bc593",
      "managed" : false
    }
  }
}

```

Here is Elasticsearch cloud version we are using:

```auto
{
  "name" : "<xyz>",
  "cluster_name" : "<xuz>",
  "cluster_uuid" : "<xyz>",
  "version" : {
    "number" : "8.0.0",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "<xyz>",
    "build_date" : "2022-02-03T16:47:57.507843096Z",
    "build_snapshot" : false,
    "lucene_version" : "9.0.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

```

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [March 28, 2022, 9:39pm UTC](https://discuss.elastic.co/t/index-life-cycle-policy-to-delete-indices/300649/4 "2022-03-28T21:39:12Z")

</div>

> [@tkotthakota](#):
>
> One thing I noticed that even though indices are getting purged I still see indices older than 3 days.

What does your policy look like?

---

<div class="post-metadata">

**Author:** ![tkotthakota](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tkotthakota/32/81695_2.png) [@tkotthakota](https://discuss.elastic.co/u/tkotthakota)\
**Post date:** [March 29, 2022, 2:35am UTC](https://discuss.elastic.co/t/index-life-cycle-policy-to-delete-indices/300649/5 "2022-03-29T02:35:39Z")

</div>

Policy I mentioned at the beginning of this thread.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [March 29, 2022, 2:43am UTC](https://discuss.elastic.co/t/index-life-cycle-policy-to-delete-indices/300649/6 "2022-03-29T02:43:47Z")

</div>

Ahh yes, sorry about that.

What does an [`_explain`](https://www.elastic.co/guide/en/elasticsearch/reference/current/ilm-explain-lifecycle.html) on the policy return.

---

<div class="post-metadata">

**Author:** ![tkotthakota](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tkotthakota/32/81695_2.png) [@tkotthakota](https://discuss.elastic.co/u/tkotthakota)\
**Post date:** [March 29, 2022, 5:59pm UTC](https://discuss.elastic.co/t/index-life-cycle-policy-to-delete-indices/300649/7 "2022-03-29T17:59:43Z")

</div>

\_explain is returning something similar to below one but for all indices.  
Also I am wondering if there a way to put index paterrn like `catalog*` to apply the ILM policy to specific indices?

```auto
{
  "indices" : {
    "catalog_1_7f64fbab-6797-4047-ac79-f50a5945ba9b_test_sha2nvq4_202203072005_4a4bc593" : {
      "index" : "catalog_1_7f64fbab-6797-4047-ac79-f50a5945ba9b_test_sha2nvq4_202203072005_4a4bc593",
      "managed" : false
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![tkotthakota](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tkotthakota/32/81695_2.png) [@tkotthakota](https://discuss.elastic.co/u/tkotthakota)\
**Post date:** [March 29, 2022, 10:18pm UTC](https://discuss.elastic.co/t/index-life-cycle-policy-to-delete-indices/300649/8 "2022-03-29T22:18:47Z")

</div>

Do we need to create index templates?

---

<div class="post-metadata">

**Author:** ![tkotthakota](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tkotthakota/32/81695_2.png) [@tkotthakota](https://discuss.elastic.co/u/tkotthakota)\
**Post date:** [March 30, 2022, 3:57pm UTC](https://discuss.elastic.co/t/index-life-cycle-policy-to-delete-indices/300649/9 "2022-03-30T15:57:29Z")

</div>

Appreciate any pointers on this issue.

---

<div class="post-metadata">

**Author:** ![tkotthakota](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tkotthakota/32/81695_2.png) [@tkotthakota](https://discuss.elastic.co/u/tkotthakota)\
**Post date:** [April 1, 2022, 1:06am UTC](https://discuss.elastic.co/t/index-life-cycle-policy-to-delete-indices/300649/10 "2022-04-01T01:06:46Z")

</div>

Hello,

I was informed that indicies created before the policy created won't be deleted automatically. Which is fine.  
However even the indices created after the policy is created also NOT getting deleted. Any dea?

-Tej

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 1, 2022, 1:59am UTC](https://discuss.elastic.co/t/index-life-cycle-policy-to-delete-indices/300649/11 "2022-04-01T01:59:27Z")

</div>

> [@tkotthakota](#):
>
> `"managed" : false`

That indicates that it has no policy attached to the index, so it won't be deleted.

What does your template that should be implementing the policy look like?

---

<div class="post-metadata">

**Author:** ![tkotthakota](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tkotthakota/32/81695_2.png) [@tkotthakota](https://discuss.elastic.co/u/tkotthakota)\
**Post date:** [April 1, 2022, 4:30pm UTC](https://discuss.elastic.co/t/index-life-cycle-policy-to-delete-indices/300649/12 "2022-04-01T16:30:08Z")

</div>

Hi Mark,

Thanks for the reply. I am not aware of the step. Looking at some other thread I found that after creating policy I need to add below two steps to attach the policy to indexes starting with the name `catalog`  
Are these steps sufficient? Is there a clear documentation on this process. I could not find one.

Also, I just added below two yesterday. Does it mean indexes created earlier won't use the policy but only newly created indexes? I ask because I do not see old indices getting purged.

How often the policy run to purge indices once its started working as expected?

```auto
PUT /catalog*/_settings?pretty
{
  "lifecycle.name": "QA_only_cleanup_old_indices"
}

PUT /_template/cleanup_policy_template?pretty
{
"index_patterns": ["catalog*"], "settings": { "index.lifecycle.name": "QA_only_cleanup_old_indices" }
}

```

---

<div class="post-metadata">

**Author:** ![tkotthakota](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tkotthakota/32/81695_2.png) [@tkotthakota](https://discuss.elastic.co/u/tkotthakota)\
**Post date:** [April 1, 2022, 4:55pm UTC](https://discuss.elastic.co/t/index-life-cycle-policy-to-delete-indices/300649/13 "2022-04-01T16:55:03Z")

</div>

This is what explain produced on one of the indices after applying above 2 commands. Indices matching are still not purged yet. Wondering if there is a time frequency when policy runs to purge these??

```auto
  "indices" : {
    "catalog_1_3f73a49a-3137-472c-xyz" : {
      "index" : "catalog_1_3f73a49a-3137-472c-xyz",
      "managed" : true,
      "policy" : "QA_only_cleanup_old_indices",
      "lifecycle_date" : "2022-03-28T06:01:05.754Z",
      "lifecycle_date_millis" : 1648447265754,
      "age" : "4.45d",
      "phase" : "delete",
      "phase_time" : "2022-04-01T16:45:34.206Z",
      "phase_time_millis" : 1648831534206,
      "action" : "delete",
      "action_time" : "2022-04-01T16:45:34.206Z",
      "action_time_millis" : 1648831534206,
      "step" : "wait-for-shard-history-leases",
      "step_time" : "2022-04-01T16:45:34.206Z",
      "step_time_millis" : 1648831534206,
      "phase_execution" : {
        "policy" : "QA_only_cleanup_old_indices",
        "phase_definition" : {
          "min_age" : "3d",
          "actions" : {
            "delete" : {
              "delete_searchable_snapshot" : true
            }
          }
        },
        "version" : 2,
        "modified_date" : "2022-03-24T23:26:17.724Z",
        "modified_date_in_millis" : 1648164377724
      }
    },

```

---

<div class="post-metadata">

**Author:** ![tkotthakota](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tkotthakota/32/81695_2.png) [@tkotthakota](https://discuss.elastic.co/u/tkotthakota)\
**Post date:** [April 1, 2022, 6:22pm UTC](https://discuss.elastic.co/t/index-life-cycle-policy-to-delete-indices/300649/14 "2022-04-01T18:22:01Z")

</div>

I see that old indices are getting deleted after attach the indices using above command.

What is the need for \_template command?  
I get below message after applying on the cluster

```auto
PUT /_template/cleanup_policy_template?pretty
{
"index_patterns": ["catalog_1_priya*"], "settings": { "index.lifecycle.name": "test_cleanup" }
}
#! Legacy index templates are deprecated in favor of composable templates.
{
  "acknowledged" : true
}

```

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 4, 2022, 12:29am UTC](https://discuss.elastic.co/t/index-life-cycle-policy-to-delete-indices/300649/15 "2022-04-04T00:29:43Z")

</div>

> [@tkotthakota](#):
>
> Also, I just added below two yesterday. Does it mean indexes created earlier won't use the policy but only newly created indexes? I ask because I do not see old indices getting purged.

[Manage existing indices | Elasticsearch Guide [8.1] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/8.1/ilm-with-existing-indices.html) applies here. But yes, you need to explicitly add those as per [Configure a lifecycle policy | Elasticsearch Guide [8.1] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/8.1/set-up-lifecycle-policy.html#apply-policy-multiple)

> [@tkotthakota](#):
>
> How often the policy run to purge indices once its started working as expected?

Every 10 minutes by default.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 2, 2022, 12:30am UTC](https://discuss.elastic.co/t/index-life-cycle-policy-to-delete-indices/300649/16 "2022-05-02T00:30:04Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.