Question about "Rollover" in Index Lifecycle Policy

[vp096 (Valentin)]

Hello to all,

I have a question when I configure my first "Index Lifecycle Policy". I am given the following option:

Is this a problem or a risk of data loss if I don't enable it? Because I want to keep my data for 365 days without ever having to modify it.

Thank you in advance for your help.

[warkolm (Mark Walkom)]

Rollover is when you create a new index, it doesn't delete the data.
There will be a section later in the policy that lets you configure when rolled indices are deleted.

[vp096 (Valentin)]

Thank's @warkolm for the reply.

Ok I see. But it's also possible to not use the rollover and just choose to delete entries older than 365 days?

For example, I keep my data for 10 days in the "warm" phase, and then I put them for 365 days in the "cold" zone before deleting them. So after 375 days my data will be deleted.

In fact it is not a problem if I do not use the rollover in my case?

[warkolm (Mark Walkom)]

If you want to do this to need to roll your indices over.
ILM doesn't delete specific entries/events/documents in an index, it deletes the indices themselves.

[vp096 (Valentin)]

Thank you @warkolm for this clarification.

I'm sorry, I'm new to Elastic, Kibana and Logstash and still some parts are not clear to me.

In practice, I push events to Elastic with Logstash, always with the same index "aruba-logs".

I need to be able to view this data for 365 days and then delete it from elastic. Can you explain me briefly how to do this?

Because if I activate the "Rollover", my index will be modified by what value?

Thank you in advance for your help.

[warkolm (Mark Walkom)]

Your best option is to have a read through ILM: Manage the index lifecycle | Elasticsearch Guide [7.14] | Elastic to get an idea of how ILM works and the concepts it uses.

Then let us know if you have further questions.

[vp096 (Valentin)]

Finally, what happens when the indexes reach this stage? Does it delete the data older than the chosen value or does it only delete the index?

[warkolm (Mark Walkom)]

It deletes the index.

[vp096 (Valentin)]

I finally find the answers in another post : "Your data resides in your index, if you delete an index, the data is also deleted and space will be freed in the disk"

Thank's for your help