How to automatically delete index data after a few days or after certain size limit

Elastic Stack Elasticsearch
1

Hello,

I have a scenario where i need to clean up the index data after a given number of days or after a certain storage size is occupied. Is there any way of achieving this without deleting the actual index?

Thanks

2

Welcome!

Deleting the index is the best and most efficient way for doing this.
With aliases, this would look like cleaning up data "without" deleting indices.

I'd start reading from here: ILM: Manage the index lifecycle | Elasticsearch Guide [8.8] | Elastic

3

@dadoonet thanks for your reply. I added an ILM policy where i have a Hot phase followed directly by Delete phase. The min_age for the delete phase is set to 30 mins, but i still see that the index remains in the Hot phase for 40 mins exact and then moves to the Delete phase where it remains for 10 mins before getting deleted.
What is the reason behind the index staying in the hot phase for 40 mins instead of the 30 mins value defined by me?
And the same for the delete phase where it remains for 10 mins before being finally deleted - is this configurable from our side?

4

ILM checks for indices that meet some policy criteria every 10 minutes per default, so it may take sometimes 2 or more check for an action to happen.

This can be configured, but I'm not sure if setting a lower value can impact the cluster or not.

2 Likes
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.