Indices not rolling over

[lchan]

I am troubleshooting an issue where my indices (filebeats*) are not rolling from hot to warm.

I changed rollover max_age=30d to max_age=1d. However, the indices of the ILM action is "completed" and looks like they won't get rolled over.

1. What is the next troubleshooting step if the ILM policy is setup correctly but still not rolling over?
2. Is there an API to rollover an index manually? I have attached a screen of the warm node setup.

GET filebeat-7.12.1-2021.06.07-000005/_ilm/explain
{
"indices" : {
"filebeat-7.12.1-2021.06.07-000005" : {
"index" : "filebeat-7.12.1-2021.06.07-000005",
"managed" : true,
"policy" : "filebeat",
"lifecycle_date_millis" : 1623243828019,
"age" : "15.17m",
"phase" : "hot",
"phase_time_millis" : 1623059628176,
"action" : "complete",
"action_time_millis" : 1623243829088,
"step" : "complete",
"step_time_millis" : 1623243829088,
"phase_execution" : {
"policy" : "filebeat",
"phase_definition" : {
"min_age" : "0ms",
"actions" : {
"rollover" : {
"max_size" : "40gb",
"max_age" : "1d"
}
}
},
"version" : 6,
"modified_date_in_millis" : 1623243785876
}
}
}
}

GET _ilm/policy/filebeat
{
"filebeat" : {
"version" : 6,
"modified_date" : "2021-06-09T13:03:05.876Z",
"policy" : {
"phases" : {
"warm" : {
"min_age" : "7d",
"actions" : {
"set_priority" : {
"priority" : 50
}
}
},
"hot" : {
"min_age" : "0ms",
"actions" : {
"rollover" : {
"max_size" : "40gb",
"max_age" : "1d"
}
}
},
"delete" : {
"min_age" : "7d",
"actions" : {
"delete" : {
"delete_searchable_snapshot" : true
}
}
}
}
}
}
}

[Wolfram_Haussig]

Hi,

Are you sure that your index did not rollover? Is there no index filebeat-7.12.1-*-000006 available?

It is possible that the index did rollover and is still in hot state as your ILM policy defines that an index should switch to warm state 7 days after rollover which would be next week.

There is a manual rollover API: Rollover API | Elasticsearch Guide [master] | Elastic

You could try POST filebeat-7.12.1/_rollover where filebeat-7.12.1 is your rollover alias.

Best regards
Wolfram

[lchan]

@Wolfram_Haussig

That's the issue, the min_age should have set to 1d. Thanks!

@lchan

min_age