How to configure datastream and ILM to keep logs for specific time

Elastic Observability Logs
1

I want to configure datastream for gathering my application logs. And I want to keep the data from the app for a specific time, let say for one week. I will not use hot-warm-cold architecture, I just want to have logs for one week and then delete it. And I'm struggling with configuring ILM for that.

If I understand it correctly I can setup rollover based on index creation date. So I can set rollover for one day and then setup delete phase with 6 days. But what will happen with index which is already rolled over ? It will still remain searchable within datastream ?

2

Hi @lancer_enkor,

that's a good question. In short: yes. A datastream is a concept in Elasticsearch, that is backed by any number of read indices and one write index. A rollover essentially creates a new empty write index and turns the previous write index into a read index. Querying the datastream will query all backing read and write indices. The datastream docs have some nice visuals to explain it better than I could with words.

Does that answer your question?

3

Yes, thank you for clarification.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.