ILM not deleting upcoming indices(ILM not working)

Elastic Stack Elasticsearch
1

Hi, I am using elk 8.0.0v in windows. Data deletion of upcoming indices is not happening for the time duration setted in ILM (ex:5mints (for testing purpose)).I have created ILM template and assigned metricbeat 8.0.0.
There is no error showing in kibana and Elasticsearch node logs.

This is the source code i implemented (it is not working):

PUT _ilm/policy/metricbeat
{
"policy": {
"phases": {
"hot": {
"min_age": "0ms",
"actions": {
"set_priority": {
"priority": 100
}
}
},
"delete": {
"min_age": "5m",
"actions": {
"delete": {
"delete_searchable_snapshot": true
}
}
}
}
}

Please help or suggest any solution.

Thanks

2

1st that is not a valid ILM with Hot and Delete only works after the rollover happens you have no rollover for hot. I would recommend using the UI in Kibana and look at what the request looks like

PUT _ilm/policy/test
{
  "policy": {
    "phases": {
      "hot": {
        "actions": {
          "rollover": {
            "max_age": "1d",
            "max_primary_shard_size": "50gb"
          },
          "set_priority": {
            "priority": 100
          }
        },
        "min_age": "0ms"
      },
      "delete": {
        "min_age": "1d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

2nd ILM is not designed for time frame on the order of minutes or exact bytes.... the background task runs every 10 mins or so. Think of ILM is meant to run on "Hours and Gigabytes" it is not mean to be exact at the minute or byte...

In the real world at scale of size and time it works as designed.

We see this often people want to test ILM with tiny shard sizes and minutes and wonder why it does not execute on the exact minute and size... it will not you can put in 1 hour or so and it will be close... etc.

1 Like
3

Hi Stephen,

The rollover needs max_primary_shard_size and max_age on the ILM definition to to trigger .

4

Not sure if that is a question?...

You need at least one of them but can also use both and rollover will happen when the first one is met.

5

sorry stephen, i forget ? mark, yes it was a question.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.