Elasticsearch Curator delete old-index after rollover?


#1

I have the following curator set-up in curator-actions.yml which is configured to delete indices based on age filter but I would like to set-up a config that deletes the old-index immediately after a rollover is successful so I am only ever left with the current index. Here is my current delete action...any help is much appreciated! :slight_smile:

    action: delete_indices
    description: >-
      Delete indices older than 5 days (based on index creation date)
    options:
      ignore_empty_list: True
      continue_if_exception: True
    filters:
    - filtertype: pattern
      kind: prefix
      value: applogging-test
    - filtertype: age
      source: creation_date
      direction: older
      unit: days
      unit_count: 3

(Christian Dahlqvist) #2

Why would you like to remove the data so soon after rollover? That could at times leave you with an almost empty current index and no older data. How long do you need to keep the data available in the cluster?


#3

Yes, I've made a mistake in my question.

I want to keep the last N number of indices. In my case N just happens to be 2. I have read that the recommended Index size is 20GB to 40GB. I have set my rollover to occur at 30GB, so right after a rollover I will have around 30GB of data which is enough for me and why I only need to keep two indices around.


#4

SOLVED! In case someone is looking at this feed I've found a solution with the good help of @theuntergeek detailed on stackoverflow: https://stackoverflow.com/questions/53320317/elasticsearch-curator-delete-action-delete-n-older-indices-after-rollover/53353788#53353788 .

Essentially, you can achieve this with a scheduled elastic curator - rollover and delete actions. These can be scheduled frequently as rollover and delete actions will only execute IF the conditions specified are met (if not met nothing happens and no hard done!)


(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.