Hello,
I hope I can find help in the community forum for this issue I've been having since the beginning of using Elastic.
My problem statement is this: when I start using a new version of a beat (i.e. go from filebeat-7.16.1 to filebeat-7.17.1) it create a base index called "filebeat-7.17.1" to store the data, but does not have ILM enabled.
The only valid way I have found to enable ILM on such an index is to re-index the index with another name (i.e. filebeat-7.17.1 --> filebeat-7.17.1-reindex), delete the original index (i.e. filebeat-7.17.1) and run the "filebeat setup" command to enable ILM. Only then I see a new index name called "filebeat-7.17.1-2022.05.17-000001" with ILM enabled on it. In order to not lose the original data, I then merge back "filebeat-7.17.1-reindex" into the new "filebeat-7.17.1-2022.05.17-000001".
I cannot endure this process anymore. I think it might be as simple as adding some lines to the beats .yml file to enable ILM on first index creation, but I have not been successful at it yet.
Basically, I am looking for two things:
- Enable ILM on the first index on new beat version startup;
- How to fix those indices that do not have ILM enabled.
Thank you for any help you can provide to make my life easier.
Joey