Need to implement a Lifecycle, but I have daily named indicies

Elastic Stack Elasticsearch
1

I hope I'm just not seeing something. I have version 7.4 of the Elastic Stack
Logstash creates indices as:
output {
elasticsearch {
action => "index"
hosts => [ "localhost:9200" ]
index => "memleak-%{+YYYY.MM.dd}"
workers => 1
}
for 9 different inputs (the 'memleak' part of the index name changes for each input, but otherwise the same)

So, I get a new index every day. Now I am finding I need to implement a lifecycle policy so I can age out old data.
I created a simple policy to delete data after 45 days of creation.
But I have to apply this policy to each index. And since a new indices are created everyday, I have to apply the policy to the new indices everyday?

2

I think you can apply your ILM policy via a template. More than one template can be applied at index creation (in ascending priority order).

For example, if you had an ILM policy "delete-at-45" that just had a delete phase, you could create a template with just that ILM setting but index patterns matching "memleak-*" and it's relation. Any other templates would still contribute their mappings and settings.

ILM is designed for more than this (rollover), but I think you can do this subset of ILM function.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.