Enabling ILM after index exists / Enable ILM on first index creation with new version

Elastic Stack Elasticsearch
1

Hello,

I hope I can find help in the community forum for this issue I've been having since the beginning of using Elastic.

My problem statement is this: when I start using a new version of a beat (i.e. go from filebeat-7.16.1 to filebeat-7.17.1) it create a base index called "filebeat-7.17.1" to store the data, but does not have ILM enabled.

The only valid way I have found to enable ILM on such an index is to re-index the index with another name (i.e. filebeat-7.17.1 --> filebeat-7.17.1-reindex), delete the original index (i.e. filebeat-7.17.1) and run the "filebeat setup" command to enable ILM. Only then I see a new index name called "filebeat-7.17.1-2022.05.17-000001" with ILM enabled on it. In order to not lose the original data, I then merge back "filebeat-7.17.1-reindex" into the new "filebeat-7.17.1-2022.05.17-000001".

I cannot endure this process anymore. I think it might be as simple as adding some lines to the beats .yml file to enable ILM on first index creation, but I have not been successful at it yet.

Basically, I am looking for two things:

  1. Enable ILM on the first index on new beat version startup;
  2. How to fix those indices that do not have ILM enabled.

Thank you for any help you can provide to make my life easier.

Joey

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.