Delete all indices exept some based on the index name

Elastic Stack Elasticsearch
1

Hello all,
We are using curator to delete indices after a certain amount of time in our environments. I am facing an issue with understanding how to make it work for deleting indices based on time but leaving some indices which have a particular string in the name.
For example : Delete all indices older than 4 weeks except the indices which have "secure" in it's name.
Here is my configuration :

---
actions:
  1:
    action: delete_indices
    description: >-
      Delete indices older than 4 weeks (based on index name).
    options:
      ignore_empty_list: True
      timeout_override:
      continue_if_exception: True
      disable_action: False
    filters:
    - filtertype: pattern
      kind: regex
      value: '*secure*'
      exclude: True
    - filtertype: pattern
      kind: regex
      value: '*loyalty*'
      exclude: True
    - filtertype: age
      source: name
      direction: older
      timestring: '%Y.%W'
      unit: weeks
      unit_count: 4
      exclude:

Here is the error I get when I run it.

Failed to complete action: delete_indices.  <class 'sre_constants.error'>: nothing to repeat

Could anybody help me understand how to do it or what am I doing wrong here?
Thank you.

2

I found out what the issue was. It was with regex. I changed the regex to '.*secure*.'. I have one similar index called '.*unsecure*.'and I would like to make another action file to delete these both indices after 8 weeks instead of 4 weeks for all other indices. I am not able to combine both. When I make two filters and run curator, it gives me empty list.

3

You should immediately recognize the problem in that '.*secure*.' will also match '.*unsecure*.' because of how regular expressions work.

If you want to use the period . as part of the match, then you should escape it with a backslash. I am going on the assumption that you're looking for the final period with this example:

'.*secure.*\.' will match indexname.secure.12345, and also indexname.unsecure.23456.

However, '.*\.secure\..*' will match indexname.secure.12345 and not match indexname.unsecure.23456.

4

Hi @theuntergeek ,
Thank you for your reply. Yes, you are right about the regex. I have forgot to update the question because I have changed the index name totally, it's "loyalty". So in short, I have one index called "secure"(ex my-index.secure-2021.24) and the other one is "loyalty"(ex my-index.loyalty-2021.24). Separately both work as expected but I am not able to combine both in one action file.

5

In that case, what you're looking to do is add a logical OR to your regex pattern:

    filters:
    - filtertype: pattern
      kind: regex
      value: '*secure*|*loyalty*'
      exclude: True

(or something rather like this).

6

Hi @theuntergeek ,
Thank you again.
Yes,it can be done. I didn't think of that. My bad.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.