Hi,
I am trying to delete or archive documents older than 3 months. Since our use case allows customers to update the data for the last 3 months, so at any moment in time, we should have the last 3 months of data. According to my understanding, the ILM policy will delete the entire index after a set period (or when shard grows to a set size, etc). Suppose new data is indexed in the past 299 days, then the rollover day, the whole index will be deleted and we will not be able to update the previous data. Is there a way to handle this situation using ILM or any query?
Welcome to our community! 
What sort of data is this?
We can make updates to data in any rolling window of 3 months so it shouldn't come under time series data. Mostly contains client's data. Also it's used for transactional queries.
ILM works on the assumption that the data is immutable once it is indexed, so the removal of data is handled at the index level. If you are updating data then you can't use ILM.
That's why I asked what sort of data it is.
You will have to manage this outside of ILM. A delete by query can do it, but it's expensive.
Got it. Thank you
Hi is there any alternative to QueryBuilder(ES 5.6) in ES 8.6 ?
Hey.
That's another question. Could you please open a new discussion for this and share more details about what you are meaning?
Sure.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.