Creating Index Lifecycle Policies

Elastic Stack Kibana
#1

Hello. On ELK stack 7.6.2, I have a policy set for deleting all indices older than 30 days.

The policy was set as follows:

PUT _ilm/policy/cleanup-history
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {}
      },
      "delete": {
        "min_age": "30d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

Out of the indices that are being deleted every 30 days, I want to selectively delete an index e.g beginning with uat_sd_taction* every 5 days as it tends to grow big very fast.

Please guide me how can I set that option?

Thanks

#2

You'll want to do the following:

  • Create a second Index Lifecycle Policy to delete an index every 5 days
  • Create an Index Template to apply your second Index Lifecycle Policy using the index pattern uat_sd_taction*
1 Like
#3

Thanks. So I would presume I could put more than one comma separated index patterns in the template as below. So essentially all the ones that the new policy applies on:


PUT _template/taction_template
{
  "index_patterns": ["test-*", "test1-*", "test2-*"], 
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 1,
    "index.lifecycle.name": "my_policy", 
    "index.lifecycle.rollover_alias": "test-alias" 
  }
}

Please confirm

#4

Yes, you can specify multiple index patterns like that! :grinning_face_with_smiling_eyes:

#5

Thanks.

I put an additional policy and template as follows:

PUT _ilm/policy/cleanup-crnlp
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {}
      },
      "delete": {
        "min_age": "2d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}


PUT _template/crnlp_template
{
  "index_patterns": ["uat_sd_taction1*", "uat_sd_taction2*", "uat_sd_taction3*", "uat_sd_taction4*"], 
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 1,
    "index.lifecycle.name": "cleanup-crnlp",
    "index.lifecycle.rollover_alias": "crnlp-alias"
    }
}

I chose the index name correctly (followed by an asterisk). For some reason the new policy does not seem linked to any indices and also I don't notice indices older than 2 days getting deleted. See below:

image
image1199×257 20.2 KB

Please advise. Do the old indices start getting deleted right away?

#6

Sorry for the confusion. The index template is only used when new indices are created.

You will need to manually apply your new index lifecycle policy to any existing indices.

See also: Manage existing indices.

#7

Thanks for pointing out the manually applying lifecycle policy to existing indices page. So I created a policy named cleanup-crnlp as shown above. Now to add existing indices shall I replace test-index below with my index name? Also my indices are rolling with date and their pattern has names such as uat_sd_taction1*

PUT test-index
{
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 1,
    "index.lifecycle.name": "cleanup-crnlp"
  }
}

In a nutshell, how do I introduce my existing indices (with pattern as uat_sd_taction1*) to the new policy?

Thanks

#8

This should do it:

PUT uat_sd_taction1*/_settings 
{
  "index": {
    "lifecycle": {
      "name": "cleanup-crnlp"
    }
  }
}