Purpose of "Data Retention" in index template

Elastic Stack Elasticsearch
1

Hi,

Can someone explain to me what the purpose of the data retention field is in an index template?
My understanding is that the ILM is responsible for moving data between phases, rollover and optionally deletion. What is the effect of setting e.g. 3 hours data retention on the index template but then setting 1 day on the delete phase in the relevant ILM?

Thanks

2

The template.lifecycle. data_retention field is how you configure the new Datastream Lifecycle management (DLM), an alternative to ILM that is built right into the datastream settings: Data stream lifecycle | Elasticsearch Guide [8.16] | Elastic

You do not have to use it but it can simplify management in many use-cases.

If a datastream is covered under both an ILM policy and there is DLM configured, only one system will work at a time on each underlying index as explained here: Tutorial: Migrate ILM managed data stream to data stream lifecycle | Elasticsearch Guide [8.16] | Elastic -- you can use the setting index.lifecycle.prefer_ilm to pick the winner if you are applying both.

3

great stuff, thx

1 Like