I have the following curator set-up in curator-actions.yml which is configured to delete indices based on age filter but I would like to set-up a config that deletes the old-index immediately after a rollover is successful so I am only ever left with the current index. Here is my current delete action...any help is much appreciated!
action: delete_indices
description: >-
Delete indices older than 5 days (based on index creation date)
options:
ignore_empty_list: True
continue_if_exception: True
filters:
- filtertype: pattern
kind: prefix
value: applogging-test
- filtertype: age
source: creation_date
direction: older
unit: days
unit_count: 3
Why would you like to remove the data so soon after rollover? That could at times leave you with an almost empty current index and no older data. How long do you need to keep the data available in the cluster?
I want to keep the last N number of indices. In my case N just happens to be 2. I have read that the recommended Index size is 20GB to 40GB. I have set my rollover to occur at 30GB, so right after a rollover I will have around 30GB of data which is enough for me and why I only need to keep two indices around.
Essentially, you can achieve this with a scheduled elastic curator - rollover and delete actions. These can be scheduled frequently as rollover and delete actions will only execute IF the conditions specified are met (if not met nothing happens and no hard done!)
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.