ILM policy don't delete the index even though delete phase is applied

Elastic Observability
1

Hello All,

I have appled ILM policy to the Index and policy is defined to go from hot phase: 1day,1gb max (rollover) to delete phase after 1day from rollover,The issue I am facing is that the Index dont get deleted and shows complete satate.
(HOT->DELETE)
Any Suggestion?

PUT _ilm/policy/mis-metric-policy
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "rollover": {
            "max_age": "1d",
            "max_size": "1gb"
          },
          "set_priority": {
            "priority": null
          }
        }
      },
      "delete": {
        "min_age": "1d",
        "actions": {
          "delete": {
            "delete_searchable_snapshot": true
          }
        }
      }
    }
  }
}

For new Index created also these policy dont get applied even after updating the policy!!
How can these be tested or resolved?

{
  "indices" : {
    "mis-metric-2022.04.20-000059" : {
      "index" : "mis-metric-2022.04.20-000059",
      "managed" : true,
      "policy" : "mis-metric-policy",
      "lifecycle_date_millis" : 1650503149638,
      "age" : "1.52d",
      "phase" : "hot",
      "phase_time_millis" : 1650434750263,
      "action" : "complete",
      "action_time_millis" : 1650503150394,
      "step" : "complete",
      "step_time_millis" : 1650503150394,
      "phase_execution" : {
        "policy" : "mis-metric-policy",
        "phase_definition" : {
          "min_age" : "0ms",
          "actions" : {
            "rollover" : {
              "max_size" : "1gb",
              "max_age" : "1d"
            },
            "set_priority" : {
              "priority" : null
            }
          }
        },
        "version" : 8,
        "modified_date_in_millis" : 1650353898387
      }
    }
  }
}

"action" and "step" is complete and "phase" hot,will this now go to delete state?....In my case it dosent

image
image930×287 23 KB

Thanx,

2

Your policy says delete 1 day after rollover.. which is 1 day

1day (rollover) + delete 1 day after rollover = 2 days total

From the status

"age" : "1.52d",

So it has not reached the delete phase yet.

As to why the ILM is not applied you would need to share your template where the ILM is specified.

3

Thanx for your quick response @stephenb !!

Here is the template,the new Index that are getting created also not following the updated policy,by that I mean from hot to I see complete phase but never index never gets deleted which is incraesing the disk space.

PUT _index_template/mis-metric
{
  "template": {
    "settings": {
      "index": {
        "lifecycle": {
          "name": "mis-metric-policy",
          "rollover_alias": "mis-metric"
        },
        "number_of_shards": "1",
        "number_of_replicas": "0"
      }
    },
    "mappings": {
      "_routing": {
        "required": false
      },
      "numeric_detection": false,
      "dynamic_date_formats": [
        "strict_date_optional_time",
        "yyyy/MM/dd HH:mm:ss Z||yyyy/MM/dd Z"
      ],
      "_source": {
        "excludes": [],
        "includes": [],
        "enabled": true
      },
      "dynamic": true,
      "date_detection": true,
      "properties": {
        "system.cpu.iowait.norm.pct": {
          "scaling_factor": 1000,
          "type": "scaled_float"
        },
        "system.process.memory.rss.pct": {
          "coerce": true,
          "index": true,
          "scaling_factor": 1000,
          "ignore_malformed": false,
          "store": false,
          "type": "scaled_float",
          "doc_values": true
        },
        "system.diskio.write.bytes": {
          "type": "long"
        },
        "system.load.cores": {
          "type": "long"
        },
        "system.process.cpu.total.value": {
          "type": "long"
        },
        "service.type": {
          "type": "keyword"
        },
        "system.diskio.iostat.service_time": {
          "type": "float"
        },
        "system.memory.actual.free": {
          "type": "long"
        },
        "system.cpu.idle.norm.pct": {
          "scaling_factor": 1000,
          "type": "scaled_float"
        },
        "agent.name": {
          "type": "keyword"
        },
        "system.memory.total": {
          "type": "long"
        },
        "system.diskio.write.time": {
          "type": "long"
        },
        "process.ppid": {
          "type": "long"
        },
        "system.diskio.iostat.write.await": {
          "type": "float"
        },
        "system.fsstat.total_size.free": {
          "type": "long"
        },
        "system.process.state": {
          "type": "keyword"
        }
      }
    }
  },
  "index_patterns": [
    "mis-metric-*"
  ],
  "composed_of": []
}
type or paste code here

What can be done to so that index gets deleted correctly?

Thanx,
Prashant

4

Did you try

      "delete": {
        "min_age": "0d",

That would mean delete after hot

Also if you keep updating the policy ... does not mean it will take affect immediately it depends what phase and state the index is in... so what I am saying is that change may not affect indices that have already been rolled over...

The template looks pretty good at a glance