Deleting Indexs that are X day old

Elastic Stack Elasticsearch
1

As a requirement I am trying to delete indexes that are, lets say 2 or 3 days old. I think it would just be simple to write a cron job to simply do this type of task. But is there anything ES provides that can delete indexes as such? Or is there a GUI plugin that allows you to delete old indexes in this manner? All my indexes are in this format

logstash-dd.MM.YYYY-TYPE

So they are pretty consistent.

2

Take a look at Curator, It was built specifically for doing these kinds of tasks on time series indices. You will still need to run a cron job but it simplifies the command greatly

3

The official documentation is at https://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.html

It was written specifically for this use case.

4

@theuntergeek and @colings86

Didn't even know this existed. I just want to check if this is right approach or common way to this problem:
Its not that intuitive with the link you provided but I found this which is straight forward just not sure of the version compatibility since I am on ES 1.5.1

Elasticsearch Curator -- Version 1.1.0 Released | Elastic Blog

From this link it's as simple as

curator delete --older-than 2

But the link by @theuntergeek it seems to be more like

curator delete indices --time-unit hours --older-than 2 --timestring '%Y.%m.%d.%H' --prefix logstash --suffix -server

My indexes are actually logstash-YYYY.mm.DD-{type}... I have 5 types so I should probably use a regex

And like @colings86 said use cron at the end of the day

Just out of curiosity can v1.1.0 (above) delete indices that aren't even indexed as logstash-YYYY.MM.dd
So for example I index everything without timestamps like logstash-{type} so it will just delete documents older then 2 days. I ask because I read somewhere that too many indices really hurts performance. More indices == more shards. Something I want to keep in mind.

5

That blog post is old, go by the docs that Aaron linked to as things have changed.

Curator does require indices with some time indicator attached to their name, ie it doesn't look at the index creation date, it looks at the name of the index and figures things out from that.