Index Lifecycle Management rollover on time, but deleting every 10min

[jbalaguero (Joan Balagueró)]

Hello,

We have this setting in our elasticsearch.yml configuration file:
indices.lifecycle.poll_interval: 30s

And we have a set of indexes that must roll over every 3 minutes and must be alive for 6 minutes.
The rollover is happennig on time, but the deletion only occurs exactly every 10min.

For example:
01:52:30: index idx-000001 created
01:55:55: index idx-000002 created
01:59:25: index idx-000003 created
02:02:30: index idx-000001 deleted

So it seems the 'indices.lifecycle.poll_interval' setting correctly informs about rollover actions every 30s, but not about delete actions (that is taking 10min, the setting default value).

Is this possible? Must I set up another setting for delete actions? Or am I doing something wrong?

Thanks,

Joan.

[dakrone (Lee Hinman)]

@jbalaguero

roll over every 3 minutes and must be alive for 6 minutes

Can you share your actual policy? It's a little harder to parse from this sentence, but I suspect that you have a min_age of 6 minutes in your delete phase, which means that it will be deleted 6 minutes after the rollover has been performed (so 3 + 6 = ~9 minutes total lifetime)

[jbalaguero (Joan Balagueró)]

Hi dakrone,

Yes, you are right. That's the point, deletion takes rollover as initial point and not the index creation.

I need to delete indexes taking the index creation as a base. So I have decided to delete them manually. I understand that deleting indexes manually does not interfiere in the ILM.

Thanks,

Joan.

[dakrone (Lee Hinman)]

@jbalaguero

I need to delete indexes taking the index creation as a base.

You can do this by setting the origination date for the index, which is the date used for any ILM action, see: Index lifecycle management settings in Elasticsearch | Elasticsearch Guide [8.11] | Elastic (the setting is called index.lifecycle.origination_date)