Create the correct ILM policy

Elastic Stack Kibana
1

Hello to the community!
I'm new to the stack and I have a small PROD deployment (approx 50 network devices syslog logs severity 0-4. Not sure yet about the number of hits/day ) for my client. Right now I only have one node on my cluster.

We did a POC before (by someone who left the company) and at that time he used curator to manage index. But I would like to use the build-in ILM functionality to simplify operations.
Here's the requirement :

Logs are kept at least 1 year. Audit trails regarding the last 3 months must be immediately available through the interface.

For now, I'm still on the testing phase and I decided to create a daily index (logstash output section). I have been struggling a bit on the ILM interface and I'm wondering if daily index is such a good idea. I thought about a hot phase with a maximum age (90days) then a delete phase but I see that the delete one is linked to rollover...but I don't see the use for it!
Reading the documentation and googling it somehow made things even more confusing :frowning:

Thanks again for the help!

2

Hi @kastu707,

Thanks for reaching out.

This blog article might be able to help you Implementing Hot-Warm-Cold in Elasticsearch with Index Lifecycle Management | Elastic Blog
It gives an example on ILM using the API as well as through the UI.

I thought about a hot phase with a maximum age (90days) then a delete phase but I see that the delete one is linked to rollover...but I don't see the use for it!

By specifying a max_age for the hot phase in the UI, you are enabling rollover. If rollover was not enabled on the hot phase, deletion would be based on days/hours from index creation. Also, this post details some of the benefits of the rollover pattern: https://www.elastic.co/blog/managing-time-based-indices-efficiently

For further questions about ILM the elasticsearch forum might be a better place to ask.

3

Thanks a lot @flash1293, I will have a look on it!