Elasticsearch Curator Filter Failure

Elastic Stack Elasticsearch
1

This is our current curator actionfile:

actions:
  1:
    action: delete_indices
    description: "Delete indices over 7 days old."
    options:
      ignore_empty_list: True
    filters:
    - filtertype: age
      source: name
      direction: older
      timestring: '%Y.%m.%d'
      unit: days
      unit_count: 7
      exclude: False
    - filtertype: pattern
      kind: regex
      value: '^.*string1*'
      exclude: True
    - filtertype: pattern
      kind: regex
      value: '^.*string2*'
      exclude: True
    - filtertype: pattern
      kind: regex
      value: '^.*string3*'
      exclude: True

So the idea is that this should delete all indices OVER 7 days old EXCEPT indices containing string1, string2, or string3. However, running a _cat/indices on my cluster, I am seeing a number of old indices which don't contain those string values, but curator returns this:

2017-02-14 03:27:47,968 INFO      Preparing Action ID: 1, "delete_indices"
2017-02-14 03:27:47,977 INFO      Trying Action ID: 1, "delete_indices": Delete indices over 7 days old.
2017-02-14 03:27:48,055 INFO      Skipping action "delete_indices" due to empty list: <class 'curator.exceptions.NoIndices'>
2017-02-14 03:27:48,055 INFO      Action ID: 1, "delete_indices" completed.
2017-02-14 03:27:48,055 INFO      Job completed.

I am 100% sure that none of those indices names contain the values "string1", "string2", or "string3", and all of them are named using the appropriate date-naming convention, ex. x-staging-logs-2017.01.16.

This is my curator.yml file:

client:
  hosts:
    - HOST_IP
  port: 9200
  timeout: 900
  master_only: True

Is my curator filter written correctly for my goal?? It appears to me to be so, and I thought it had worked in the past but I'm unclear as to why it is not functioning now. Are there other checks I can run to see more in-depth what is happening when I run curator??

Thank you!

2

Filters in Curator are connected/chained together by logical ANDs. Effectively, this is filtering everything out.

3

The first filter is Exclude: false, which INCLUDES all indices over 7 days old. The other three are string-based exclusions to filter particular indices from that list of deletes.

It was in fact a bad regex on our end :frowning:. We used:

- filtertype: pattern
  kind: regex
  value: '^.*string1*'
  exclude: True

when we should have had:

- filtertype: pattern
  kind: regex
  value: '^.*string1.*'
  exclude: True

After applying that fix, it worked. Sorry to take your time, but thank you!!

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.