How to guarantee exact retention time for logs?

[Kosodrom (Kosmodrom)]

Hi folks,

I have following question:

I store logs in Elasticsearch cluster and want to guarantee the promised retention time of 30 days to the developer.

I have configured my Lifecycle policy to rollover after 30 days and then delete immediately. But this lead (obviously) to the situation that when the 30 days from rollover were reached - lets say - yesterday, the index will be deleted and the developer only sees logs from 1 day.

One way to handle this would be to configure the policy to be always twice as much as the desired retention time. For example if you want to have the logs searchable for last 30 days you need to configure for example a rollover each 30 days and delete after 30 days. Am I correct?

Is there a more efficient way to guarantee to a developer a retention time of 30 days so he is always able to see the logs from now 30 days back

Thank you

[warkolm (Mark Walkom)]

Can you share your policy for this?

[Christian_Dahlqvist]

As ILM manages retention by deleting complete indices I would recommend setting the max time frame a single index covers to maximum one tenth of the total retention period. Having monthly indices with a 1 month retention period does not sound suitable.