The age of index which is using ilm policy management

Ages are calculated based off of index creation time prior to a rollover, and the time the index was rolled over after the rollover has occurred. (In policies with no rollover the index creation time is used).

To explain why, here's an example. Imagine that you have a max_age of 30d and max_size of 50gb, in a rollover action followed immediately with a delete phase with a min_age of 7d.

That means that documents in the index can range between 0 seconds and 30 days old (assuming the size condition is not meant). So when the index is deleted the documents are between 7 days and 37 days old. This is the current behavior.

Now, imagine that we did not calculate the age based on the rollover time. Imagine that we indexed documents constantly over a 5 day period, after 5 days, the index reaches 50gb and rolls over. This means that documents inside of the index are between 0 and 5 days old. ILM then deletes the index 2 days later (because it has reached an age of 7 days). This means that we delete the index containing documents that are between 2 days and 7 days old (even though the index is older).

We prefer to err on the side of keeping indices slightly longer but ensuring that retention is always at least as long as the min_age of the delete phase.

If you want to avoid this, you can use the index.lifecycle.origination_date setting to specify what time all the ILM phase transitions should use for the age.

Hopefully this helps.

1 Like