When do delete policies in ILM actually get applied?

Elastic Stack Elasticsearch
1

Hello

We have a ILM policy that deletes logs older than 240 days.

We also have a monitoring system that watches the mount point where the logs get stored for space.

Every 1-2 months, we get a alert that it gets near its threshold but randomly the next hours/days , it gets away from it but then meets it again. We imagine this is happening because it gets filled up, the ILM policy kicks in clearing logs but then in x timeframe, logs keep coming in and it fills up again.

I would like to be able to explain how the arch works when it comes to ILM and deleting: What process runs in this policy? Is it EXACTLY 240 days or is there a "refresh period" to where it looks at the policy THEN applies it? etc

Thank you

2

@riahc3 You need to check the policy details defined in your setup , if its default or changed as per your use case.

To view the logs policy in Kibana:

  1. Open the menu and go to Stack Management > Index Lifecycle Policies .
  2. Select the logs policy.

The logs policy uses the recommended rollover defaults: Start writing to a new index when the current write index reaches 50GB or becomes 30 days old.

Refer page for more details on the same.

3

According to the documentation the ILM checks for indices that match any of the conditions every 10 minutes.

What does your policy looks like? Are you deleting only by time? Please, share it.

4 
{
  "logs-ilp" : {
    "version" : 4,
    "modified_date" : "2021-09-01T09:06:02.218Z",
    "policy" : {
      "phases" : {
        "hot" : {
          "min_age" : "0ms",
          "actions" : {
            "set_priority" : {
              "priority" : 100
            }
          }
        },
        "delete" : {
          "min_age" : "210d",
          "actions" : { }
        }
      }
    }
  }
}
5

That would be the policy @leandrojmp

6

Yes, I am currently using a custom made policy for my logs.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.