Indices not being deleted with the ILM policy set

Elastic Stack Kibana
1

Hello,

I have set an ILM policy for filebeat index as below. The purpose of implementing the ILM policy was because of the huge disk space utilization because of the indices data.

    Hot phase (Required)
    Rollover: enabled
    Maximum primary shard size: 10G
    Maximum index size: 10G
    Maximum age: 7 days

    Delete phase
    Move data into phase when:10 days old

Note: The ILM policy was implemented on 29th of Dec 2023. I was expecting the policy will run smoothly as the 7 days time period has already elapsed. However, I don't see its working as expected and deletion is not happening.

image
image1152×472 39.7 KB

image
image1144×290 24.5 KB

When I do an _ilm/explain on the index, it shows as follows

GET /<filebeat-7.17.15>/_ilm/explain?human

{
  "indices" : {
    "filebeat-7.17.15-2024.01.05-000003" : {
      "index" : "filebeat-7.17.15-2024.01.05-000003",
      "managed" : true,
      "policy" : "filebeat",
      "lifecycle_date" : "2024-01-05T11:23:13.476Z",
      "lifecycle_date_millis" : 1704453793476,
      "age" : "4.09d",
      "phase" : "hot",
      "phase_time" : "2024-01-05T11:23:13.731Z",
      "phase_time_millis" : 1704453793731,
      "action" : "rollover",
      "action_time" : "2024-01-05T11:23:13.931Z",
      "action_time_millis" : 1704453793931,
      "step" : "check-rollover-ready",
      "step_time" : "2024-01-05T11:23:13.931Z",
      "step_time_millis" : 1704453793931,
      "phase_execution" : {
        "policy" : "filebeat",
        "phase_definition" : {
          "min_age" : "0ms",
          "actions" : {
            "rollover" : {
              "max_size" : "10gb",
              "max_primary_shard_size" : "10gb",
              "max_age" : "7d"
            }
          }
        },
        "version" : 6,
        "modified_date" : "2024-01-09T07:11:44.455Z",
        "modified_date_in_millis" : 1704784304455
      }
    },
    "filebeat-7.17.15-2023.12.29-000002" : {
      "index" : "filebeat-7.17.15-2023.12.29-000002",
      "managed" : true,
      "policy" : "filebeat",
      "lifecycle_date" : "2024-01-05T11:23:13.531Z",
      "lifecycle_date_millis" : 1704453793531,
      "age" : "4.09d",
      "phase" : "hot",
      "phase_time" : "2023-12-29T11:13:13.703Z",
      "phase_time_millis" : 1703848393703,
      "action" : "complete",
      "action_time" : "2024-01-05T11:23:13.931Z",
      "action_time_millis" : 1704453793931,
      "step" : "complete",
      "step_time" : "2024-01-05T11:23:13.931Z",
      "step_time_millis" : 1704453793931,
      "phase_execution" : {
        "policy" : "filebeat",
        "phase_definition" : {
          "min_age" : "0ms",
          "actions" : {
            "rollover" : {
              "max_size" : "10gb",
              "max_primary_shard_size" : "10gb",
              "max_age" : "7d"
            }
          }
        },
        "version" : 6,
        "modified_date" : "2024-01-09T07:11:44.455Z",
        "modified_date_in_millis" : 1704784304455
      }
    }
  }
}

Running version 7.17.15

Am I missing anything here? Please suggest.

Thanks,

2

Hi,

In your case, the min_age for the delete phase is 10 days. This means that 10 days after the rollover, the index should be deleted.

Regards

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.