Delete index at a regular basis

Elastic Stack Elasticsearch
1

I have made different kinds of indexes into ES and everytime I want to delete some old indexes, I need to write the currator command to delete old indices. Even though I have read through the examples from https://www.elastic.co/guide/en/elasticsearch/client/curator/current/examples.html, I still do not quite understand how can I follow them. Would anyone can tell me how can I make ES to regularly delete old indices automatically?

2

The page you link to contains an example of how to delete old indexes: https://www.elastic.co/guide/en/elasticsearch/client/curator/current/examples.html#ex_delete_indices

If this doesn't work for you please describe in what way it doesn't work. Providing Curator logs would probably make it easier to help.

3

I am originally using curator 3.5.1 but then I have upgraded to curator 4. However I could not find where curator is stored. Therefore, I would wanna know where will the configuration file be and why there will be a missing of action_file?

Here aresome screenshots for reference:


Great thanks in advance for telling where would the file be.

4

Please don't post screenshots of text that can be copied and pasted.

The action file is a YAML file containing the actions to execute, as found in the examples we talked about earlier. You can store your action file anywhere.

5

Sorry for the screenshots part.

Does that mean I need to create a new .curator directory and also my own curator.yml and action.yml file?

6

You can place curator.yml and action.yml anywhere you want and give them any name, but if you call the configuration file ~/.curator/curator.yml you won't have to use the --config option since the default will work. See https://www.elastic.co/guide/en/elasticsearch/client/curator/4.0/configfile.html for more information about the configuration file.

7

Hi @magnusbaeck,

Thanks for your answer and sorry for a late reply. I have tried creating the curator and action yaml file. However, following the example, it can only filter named index by typing the wanted-to-delete index name in the value field. How about if I want to delete any index that is older than 5 days? How can I do so?

And, how can I make the curator to run regularly on a daily basis?

Thanks.

8

However, following the example, it can only filter named index by typing the wanted-to-delete index name in the value field. How about if I want to delete any index that is older than 5 days? How can I do so?

I'm not sure what example you've been following, but the documentation's delete_indices example, Examples | Curator Reference [4.0] | Elastic, shows how to delete indices older than 45 days.

And, how can I make the curator to run regularly on a daily basis?

Use cron.

9

Hi @magnusbaeck
Thanks for the reply. I have followed exactly the same link where you embedded but from that example, it can only delete with index that start with logstash-. Thus, I would wonder how to delete any index that older than 5 days?

10

This part makes sure the deletion action only applies to indices older than 45 days:

- filtertype: age
  source: name
  direction: older
  timestring: '%Y.%m.%d'
  unit: days
  unit_count: 45
  exclude:

Over and out.

11

Hi @magnusbaeck
Thanks. I am able to solve it now.