Enabling ILM after index exists / Enable ILM on first index creation with new version


I hope I can find help in the community forum for this issue I've been having since the beginning of using Elastic.

My problem statement is this: when I start using a new version of a beat (i.e. go from filebeat-7.16.1 to filebeat-7.17.1) it create a base index called "filebeat-7.17.1" to store the data, but does not have ILM enabled.

The only valid way I have found to enable ILM on such an index is to re-index the index with another name (i.e. filebeat-7.17.1 --> filebeat-7.17.1-reindex), delete the original index (i.e. filebeat-7.17.1) and run the "filebeat setup" command to enable ILM. Only then I see a new index name called "filebeat-7.17.1-2022.05.17-000001" with ILM enabled on it. In order to not lose the original data, I then merge back "filebeat-7.17.1-reindex" into the new "filebeat-7.17.1-2022.05.17-000001".

I cannot endure this process anymore. I think it might be as simple as adding some lines to the beats .yml file to enable ILM on first index creation, but I have not been successful at it yet.

Basically, I am looking for two things:

  1. Enable ILM on the first index on new beat version startup;
  2. How to fix those indices that do not have ILM enabled.

Thank you for any help you can provide to make my life easier.


This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.