Index names when using ILM

[drolfe (Daniel)]

Hi All,

I"m currently using stack 7.1.1, I've having trouble getting logstash to append the YYYY-MM-DD to indexes after enabling Index Lifecycle Managment (ILM)

My logstash output looks like the below:

output {
        elasticsearch {
                hosts => "192.168.10.11"
                index => "logstash-%{+YYYY.MM.dd}"
                ilm_enabled => true
                ilm_rollover_alias => "logstash-ilm"
                ilm_pattern => "000001"
                ilm_policy => "logstash-policy"
        }
}

However my indexes look like the below:

logstash-ilm-000001
logstash-ilm-000002
logstash-ilm-000003

Metricbeat seems to be using the names format I"m after:

metricbeat-7.1.1-2019.06.27-000001
metricbeat-7.1.1-2019.06.28-000001
metricbeat-7.1.1-2019.06.29-000001

Any help would be great

[Christian_Dahlqvist]

As you are using rollover, a sequence number must be added as multiple indices can be created per day. This is why you get a sequence number in addition to the date.

[drolfe (Daniel)]

Hi Thanks for the reply,

I know what the ILM index pattern is, what I don't understand is why I"m not getting the date in the index name

looking to go from this:

logstash-ilm-000001
logstash-ilm-000002
logstash-ilm-000003

to this:

logstash-2019.06.27-000001
logstash-2019.06.28-000001
logstash-2019.06.29-000001

Thanks Daniel

[Christian_Dahlqvist]

It is not the ILM index pattern, it is due to you using rollover. As far as I know you can use ILM with standard daily indices that do not have the incremental suffix. When using rollover it is generally not required to have a date in the index name as both ILM and Curator can handle aging based on the index data and do not need the name. If you want to have date in name and use rollover I do believe the sequence number will continually increase and not reset every day as rollover does not consider the date part of the name.