Rollover goes beyond the condition

[Mohammad.ali]

Hello every one
As the number of out indecies are growing we are considered to use ilm in elasticsearch.

this is an example of ilm we are using for test:

{
"policy": {
"phases": {
"hot": {
"min_age": "0ms",
"actions": {
"rollover": {
"max_size": "2mb"
}
}
},
"delete": {
"min_age": "1h",
"actions": {
"delete": {}
}
}
}
}
}

According to this ilm policy if the index size become more than 2mb it must be rolled over.but i does not happen and most of the time the rollover happend when the size of index is twice more than what we wrote in our condition.so imagine if we set the condition to 50G then it will rollover in 100G which is definitely a huge problem.

this is the ilm explain

{
"indices" : {
"filebeat-7.3.0-2019.11.12-000001" : {
"index" : "filebeat-7.3.0-2019.11.12-000001",
"managed" : true,
"policy" : "filebeat-7.3.0",
"lifecycle_date_millis" : 1573534932991,
"phase" : "hot",
"phase_time_millis" : 1573534933480,
"action" : "rollover",
"action_time_millis" : 1573534943518,
"step" : "check-rollover-ready",
"step_time_millis" : 1573534943518,
"phase_execution" : {
"policy" : "filebeat-7.3.0",
"phase_definition" : {
"min_age" : "0ms",
"actions" : {
"rollover" : {
"max_size" : "2mb"
}
}
},
"version" : 6,
"modified_date_in_millis" : 1571745726644
}
}
}
}

[stephenb (Stephen Brown)]

Hi @Mohammad.ali

ILM it's not optimized for such small conditions. It will also not roll over on an exact document count.

The policy triggers the next phase to happen but it does not trigger at the expense of all other workloads, it is to intended to be graceful.

When you use real or normal conditions such as GBs days or hours etc the percentage of overrun will be realistic / very small percentage.

With such a tiny use case / policy it gives the appearance that the overrun and will be a large percentage it will not be with real production value. You can use small numbers like you did just to test that your actions are correct but then you need to also test with the larger more realistic values, I suspect you will then see the desired behavior.

Example if you set a policy to roll over at 20GB it's not going to roll over at 40GB, it will roll over at 20 GB plus a very small amount.

Give it a try.