Apply ILM on multiple different index

[kamfart]

Hi everyone.

I try to figure it out something which annoying me about ILM.
I should miss something.

Here is my understanding of Logstash and Elasticsearch philosophy :

• document, objects which will contains our data parsed by Logstash
• index, objects which will regroup documents into one accessible common name
• index template, object which will point to some index describe by a regular expression
• ilm, object which will create an index n+1 under certain conditions

For me, one of the most advantage of index template is the possibility to apply and normalyze specific contrains to different document regroup by different pattern.

For example : add one numeric field with mutate filter, into several configurations files from differents sources.
And use template to convert it has integer instead of copy/paste X times mutate + convert filter into each files.

Here is my trouble with ILM :
To apply an ILM to one template, you have to declare the field "index.lifecycle.rollover_alias".
This field can not be point by two different index. (Result : alias [my_lovely_alias] has more than one write index).
Because of that, you can not apply an ILM to one entire pattern matched by your index template.

Did I misunderstood something in the documentation ?
If I didn't, is there a way to apply the same ILM to every index ?

Best,
Kamfart

[dakrone (Lee Hinman)]

@kamfart

Here is my trouble with ILM :
To apply an ILM to one template, you have to declare the field "index.lifecycle.rollover_alias".
This field can not be point by two different index. (Result : alias [my_lovely_alias] has more than one write index).
Because of that, you can not apply an ILM to one entire pattern matched by your index template.

Did I misunderstood something in the documentation ?
If I didn't, is there a way to apply the same ILM to every index ?

Yes this is unfortunately something that currently has to be done. We're aware of the problem though, and we're working on work that will make the bootstrapping easier to set up and reuse. Data Streams is one of the major pieces of work in this area.

For now, you can apply the same ILM policy to every index, however, the alias management has to be done either manually or with separate templates.

[kamfart]

Hi Dakrone,

thanks for your reply I've started to be crazy with this, haha.

Good luck with this issue !

Kamfart.