ILM and automatic bootstrap?

Elastic Stack Elasticsearch
1

Hi,

I am struggling with ILM. I want to find out if I can change our environment to use it.

Currently we have two fields in our events, which are used for creating the index:

  • logType (defines the part of the application which logged the event (e.g. session, timer, http, ...)
  • stage (defines the stage of the system (e.g. dev, staging, prod, ...)

Logstash is currently calculating the index pattern like this: <stage>-<logType>-<YYYY.MM.DD>.
This results in index names like staging-session-2020.04.20

So currently install new stages or define new logTypes in our application, the splitting to new indices is plug and play, so nothing to do in elasticsearch or logstash. Thats nice.

But if I understand correctly, I need to bootstrap indices for using ILM, that means I have to know all stage and logType combinations before they are inserted via logstash. Is that correct?

Is there any way to bootstrap it automatically and set staging-session-2020.04.20 or at least staging-session as alias automatically?

Thanks a lot,
Andreas

2

You apply an ILM policy to an index template, so you could set the index_patterns to staging-* in said template. That should be easier as I wouldn't imagine you have changing environment names compared to the log types.

You can also set your own alias name in the same index template that has the ILM policy attached to it.

Tutorial: Automate rollover with ILM | Elasticsearch Guide [7.6] | Elastic would be worth checking out :slight_smile:

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.