# Creating Index Lifecycle Policies

**URL:** <https://discuss.elastic.co/t/creating-index-lifecycle-policies/265209>\
**Category:** Kibana\
**Tags:** ilm-index-lifecycle-management\
**Created:** [February 23, 2021, 2:14pm UTC](https://discuss.elastic.co/t/creating-index-lifecycle-policies/265209 "2021-02-23T14:14:28Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![zaeemmasood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zaeemmasood/32/102383_2.png) [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\
**Post date:** [February 23, 2021, 2:14pm UTC](https://discuss.elastic.co/t/creating-index-lifecycle-policies/265209/1 "2021-02-23T14:14:28Z")

</div>

Hello. On ELK stack 7.6.2, I have a policy set for deleting all indices older than 30 days.

The policy was set as follows:

```auto
PUT _ilm/policy/cleanup-history
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {}
      },
      "delete": {
        "min_age": "30d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

```

Out of the indices that are being deleted every 30 days, I want to selectively delete an index e.g beginning with `uat_sd_taction*` every 5 days as it tends to grow big very fast.

Please guide me how can I set that option?

Thanks

---

<div class="post-metadata">

**Author:** ![jportner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jportner/32/75692_2.png) [@jportner](https://discuss.elastic.co/u/jportner)\
**Post date:** [February 23, 2021, 9:00pm UTC](https://discuss.elastic.co/t/creating-index-lifecycle-policies/265209/2 "2021-02-23T21:00:42Z")

</div>

You'll want to do the following:

- Create a second Index Lifecycle Policy to delete an index every 5 days
- Create an [Index Template](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/set-up-lifecycle-policy.html#apply-policy-template) to apply your second Index Lifecycle Policy using the index pattern `uat_sd_taction*`

---

<div class="post-metadata">

**Author:** ![zaeemmasood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zaeemmasood/32/102383_2.png) [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\
**Post date:** [February 24, 2021, 4:01pm UTC](https://discuss.elastic.co/t/creating-index-lifecycle-policies/265209/3 "2021-02-24T16:01:09Z")

</div>

Thanks. So I would presume I could put more than one comma separated index patterns in the template as below. So essentially all the ones that the new policy applies on:

```auto

PUT _template/taction_template
{
  "index_patterns": ["test-*", "test1-*", "test2-*"], 
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 1,
    "index.lifecycle.name": "my_policy", 
    "index.lifecycle.rollover_alias": "test-alias" 
  }
}

```

Please confirm

---

<div class="post-metadata">

**Author:** ![jportner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jportner/32/75692_2.png) [@jportner](https://discuss.elastic.co/u/jportner)\
**Post date:** [February 24, 2021, 4:11pm UTC](https://discuss.elastic.co/t/creating-index-lifecycle-policies/265209/4 "2021-02-24T16:11:35Z")

</div>

> [@zaeemmasood](#):
>
> So I would presume I could put more than one comma separated index patterns in the template as below.

Yes, you can specify multiple index patterns like that! 😄

---

<div class="post-metadata">

**Author:** ![zaeemmasood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zaeemmasood/32/102383_2.png) [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\
**Post date:** [February 24, 2021, 5:34pm UTC](https://discuss.elastic.co/t/creating-index-lifecycle-policies/265209/5 "2021-02-24T17:34:16Z")

</div>

Thanks.

I put an additional policy and template as follows:

```auto
PUT _ilm/policy/cleanup-crnlp
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {}
      },
      "delete": {
        "min_age": "2d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

```

```auto
PUT _template/crnlp_template
{
  "index_patterns": ["uat_sd_taction1*", "uat_sd_taction2*", "uat_sd_taction3*", "uat_sd_taction4*"], 
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 1,
    "index.lifecycle.name": "cleanup-crnlp",
    "index.lifecycle.rollover_alias": "crnlp-alias"
    }
}

```

I chose the index name correctly (followed by an asterisk). For some reason the new policy does not seem linked to any indices and also I don't notice indices older than 2 days getting deleted. See below:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/d/d/dde541f043ffb9b454e521db061ff11776311b56.png)

Please advise. Do the old indices start getting deleted right away?

---

<div class="post-metadata">

**Author:** ![jportner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jportner/32/75692_2.png) [@jportner](https://discuss.elastic.co/u/jportner)\
**Post date:** [February 24, 2021, 5:41pm UTC](https://discuss.elastic.co/t/creating-index-lifecycle-policies/265209/6 "2021-02-24T17:41:02Z")

</div>

Sorry for the confusion. The index template is only used when new indices are created.

You will need to [manually apply your new index lifecycle policy](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/set-up-lifecycle-policy.html#apply-policy-manually) to any existing indices.

See also: [Manage existing indices](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/ilm-with-existing-indices.html).

---

<div class="post-metadata">

**Author:** ![zaeemmasood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zaeemmasood/32/102383_2.png) [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\
**Post date:** [February 24, 2021, 8:44pm UTC](https://discuss.elastic.co/t/creating-index-lifecycle-policies/265209/7 "2021-02-24T20:44:17Z")

</div>

Thanks for pointing out the manually applying lifecycle policy to existing indices page. So I created a policy named `cleanup-crnlp` as shown above. Now to add existing indices shall I replace `test-index` below with my index name? Also my indices are rolling with date and their pattern has names such as `uat_sd_taction1*`

```auto
PUT test-index
{
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 1,
    "index.lifecycle.name": "cleanup-crnlp"
  }
}

```

In a nutshell, how do I introduce my existing indices (with pattern as uat\_sd\_taction1\*) to the new policy?

Thanks

---

<div class="post-metadata">

**Author:** ![jportner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jportner/32/75692_2.png) [@jportner](https://discuss.elastic.co/u/jportner)\
**Post date:** [February 24, 2021, 9:06pm UTC](https://discuss.elastic.co/t/creating-index-lifecycle-policies/265209/8 "2021-02-24T21:06:38Z")

</div>

> [@zaeemmasood](#):
>
> In a nutshell, how do I introduce my existing indices (with pattern as uat\_sd\_taction1\*) to the new policy?

This should do it:

```auto
PUT uat_sd_taction1*/_settings 
{
  "index": {
    "lifecycle": {
      "name": "cleanup-crnlp"
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![zaeemmasood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zaeemmasood/32/102383_2.png) [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\
**Post date:** [March 1, 2021, 3:05pm UTC](https://discuss.elastic.co/t/creating-index-lifecycle-policies/265209/9 "2021-03-01T15:05:38Z")

</div>

Hi Joe,

Thanks. I applied the following:

```auto
PUT uat_sd_taction1*/_settings 
{
  "index": {
    "lifecycle": {
      "name": "cleanup-crnlp"
    }
  }
}

```

It seems that this took effect for the time being as today I see old (over 2 days old) indices again.

Please advise on how can the individual index removal policy could be made permanent.

Thanks

---

<div class="post-metadata">

**Author:** ![jportner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jportner/32/75692_2.png) [@jportner](https://discuss.elastic.co/u/jportner)\
**Post date:** [March 1, 2021, 5:18pm UTC](https://discuss.elastic.co/t/creating-index-lifecycle-policies/265209/10 "2021-03-01T17:18:08Z")

</div>

> [@zaeemmasood](#):
>
> Please advise on how can the individual index removal policy could be made permanent.

If I'm understanding you correctly: you have newer indices that do not have this policy applied. Is that correct?

Looking at your template above, I think you are too specific with your index patterns. You probably need to change it to this:

```auto
PUT _template/crnlp_template
{
  "index_patterns": ["uat_sd_taction*"], 
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 1,
    "index.lifecycle.name": "cleanup-crnlp",
    "index.lifecycle.rollover_alias": "crnlp-alias"
    }
}

```

---

<div class="post-metadata">

**Author:** ![zaeemmasood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zaeemmasood/32/102383_2.png) [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\
**Post date:** [March 1, 2021, 7:52pm UTC](https://discuss.elastic.co/t/creating-index-lifecycle-policies/265209/11 "2021-03-01T19:52:44Z")

</div>

Yes the indices keep rolling with date. An example is:

```auto
uat_sd_taction-2021.02.25
uat_sd_taction-2021.02.26
uat_sd_taction-2021.02.27
uat_sd_taction-2021.02.28
uat_sd_taction-2021.03.01

```

I want to keep the most recent two and delete the rest using the following policy:

```auto
PUT _ilm/policy/cleanup-crnlp
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {}
      },
      "delete": {
        "min_age": "2d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![jportner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jportner/32/75692_2.png) [@jportner](https://discuss.elastic.co/u/jportner)\
**Post date:** [March 1, 2021, 8:31pm UTC](https://discuss.elastic.co/t/creating-index-lifecycle-policies/265209/12 "2021-03-01T20:31:11Z")

</div>

OK, then you need to use the index pattern `"uat_sd_taction*"` in your template like I mentioned above. That should behave how you want.

So you need to:

1. Updated your template accordingly
2. Manually applied the policy to any existing indices again
3. In the future, check if new indices are using the correct policy in the future with this command:

```auto
GET uat_sd_taction*/_ilm/explain

```

For more info on lifecycle progress see the [docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/getting-started-index-lifecycle-management.html#ilm-gs-check-progress).

---

<div class="post-metadata">

**Author:** ![zaeemmasood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zaeemmasood/32/102383_2.png) [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\
**Post date:** [March 22, 2021, 4:07pm UTC](https://discuss.elastic.co/t/creating-index-lifecycle-policies/265209/13 "2021-03-22T16:07:16Z")

</div>

Thanks for the help.

I still see issues and see old indices present when I run the following:

```auto
GET uat_sd_taction*/_ilm/explain

```

So far I did the following:

1. Created the following policy:

```auto
UT _ilm/policy/cleanup-crnlp
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {}
      },
      "delete": {
        "min_age": "2d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

```

The template I created (which employs this policy) is as follows:

```auto
PUT _template/2days_crnlp_template
{
  "index_patterns": ["uat_sd_taction*", "uat_md_taction*", "uat_fd_taction*", "uat_cd_taction*", "uat_dd_taction*"], 
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 1,
    "index.lifecycle.name": "cleanup-crnlp",
    "index.lifecycle.rollover_alias": "crnlp-alias"
    }
}

```

Despite of this in place for a few days, I still see indices "older" than 2 days existing in elasticsearch.

Please guide.

---

<div class="post-metadata">

**Author:** ![dakrone](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dakrone/32/23351_2.png) [@dakrone](https://discuss.elastic.co/u/dakrone)\
**Post date:** [March 24, 2021, 2:19pm UTC](https://discuss.elastic.co/t/creating-index-lifecycle-policies/265209/14 "2021-03-24T14:19:46Z")

</div>

Hi there,

It looks like you might have multiple templates that have different lifecycles, if the lifecycle is indeed not being set correctly.

Could you provide the output of `GET /uat_sd_taction<something>/_settings` where the index name is one of the old indices that you expect to be deleted (I'd like to see whether the policy was actually set in the index settings).

Also, could you show us the explain output from one of the older indices you expect to be deleted?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 21, 2021, 2:20pm UTC](https://discuss.elastic.co/t/creating-index-lifecycle-policies/265209/15 "2021-04-21T14:20:32Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.