Help with curator pattern value

Elastic Stack Elasticsearch
1

Hi Everyone,

I am trying to do a bulk delete and is struggling with defining pattern for it. Any help would be appreciated.

cloudtrail-test-dev-2017.09.17 vQk3HfDBRF2PBlPgLL6Kkg 3 1 26050 0 64.7mb 32.4mb
cloudtrail-test-dev-2017.09.05 ajssF3k2R_64SPkBOXskLQ
cloudtrail-test-dev-2017.09.06 NE9iiXOsQzGSRUQoGyshxA
cloudtrail-test-prod-2017.09.25 E7MbVCRtQa2LZybWSoNfYA 3 1 1985 0 10.3mb 5.2mb
cloudtrail-test-dev-2017.09.15 fg2b0SjRR66ycgM239CFCA
cloudtrail-test-prod-2017.09.19 unR-ONR_TVKbDUMk3McGIg 3 1 24924 0 64.5mb 32.2mb
cloudtrail-test-dev-2017.10.02 -PrJiZwrScCkKTg7EMrhPg 3 1 22052 0 55.2mb 27.6mb

I want to delete all the indices that has 2017.09 in it.

--
Niraj

2

This could be done with the pattern filter, using kind: regex:

filters:
- filtertype: pattern
  kind: regex
  value: '^.*2017\.09.*$'

When testing new configurations, be sure to use the --dry-run flag first, to make sure it does what you expect.

3

@theuntergeek Thanks for the quick response. Below is my curator action file.

actions:
  1:
    action: delete_indices
    description: >-
      Closes indices older than 15 days (based on index name), for custom-
      prefixed indices. Ignore the error if the filter does not result in an
      actionable list of indices (ignore_empty_list) and exit cleanly.
    options:
      ignore_empty_list: True
      continue_if_exception: False
      disable_action: False
    filters:
    - filtertype: pattern
      kind: prefix
      value: '^.*2017\.09.*$'
    filters:
    - filtertype: age
      source: name
      direction: older
      timestring: '%Y.%m.%d'
      unit: days
      unit_count: 15

When i try to run the curator using dry run i see that it also takes values from .08 month. Am i doing something wrong here?

2017-10-02 18:56:03,447 INFO      DRY-RUN: delete_indices: cloudtrail-dev-2017.08.23 (CLOSED) with arguments: {}
2017-10-02 18:56:03,447 INFO      DRY-RUN: delete_indices: cloudtrail-dev-2017.08.24 (CLOSED) with arguments: {}
2017-10-02 18:56:03,447 INFO      DRY-RUN: delete_indices: cloudtrail-dev-2017.08.25 (CLOSED) with arguments: {}
2017-10-02 18:56:03,448 INFO      DRY-RUN: delete_indices: cloudtrail-dev-2017.08.26 (CLOSED) with arguments: {}
2017-10-02 18:56:03,448 INFO      DRY-RUN: delete_indices: cloudtrail-dev-2017.08.27 (CLOSED) with arguments: {}
2017-10-02 18:56:03,448 INFO      DRY-RUN: delete_indices: cloudtrail-dev-2017.08.28 (CLOSED) with arguments: {}
2017-10-02 18:56:03,448 INFO      DRY-RUN: delete_indices: cloudtrail-dev-2017.08.29 (CLOSED) with arguments: {}
2017-10-02 18:56:03,448 INFO      DRY-RUN: delete_indices: cloudtrail-dev-2017.08.30 (CLOSED) with arguments: {}
2017-10-02 18:56:03,448 INFO      DRY-RUN: delete_indices: cloudtrail-dev-2017.08.31 (CLOSED) with arguments: {}
2017-10-02 18:56:03,448 INFO      DRY-RUN: delete_indices: cloudtrail-dev-2017.09.01 (CLOSED) with arguments: {}
2017-10-02 18:56:03,448 INFO      DRY-RUN: delete_indices: cloudtrail-dev-2017.09.02 (CLOSED) with arguments: {}
2017-10-02 18:56:03,448 INFO      DRY-RUN: delete_indices: cloudtrail-dev-2017.09.03 (CLOSED) with arguments: {}
2017-10-02 18:56:03,448 INFO      DRY-RUN: delete_indices: cloudtrail-dev-2017.09.04 (CLOSED) with arguments: {}
2017-10-02 18:56:03,448 INFO      DRY-RUN: delete_indices: cloudtrail-dev-2017.09.05 (CLOSED) with arguments: {}
2017-10-02 18:56:03,448 INFO      DRY-RUN: delete_indices: cloudtrail-dev-2017.09.06 (CLOSED) with arguments: {}
2017-10-02 18:56:03,449 INFO      DRY-RUN: delete_indices: cloudtrail-dev-2017.09.07 (CLOSED) with arguments: {}
2017-10-02 18:56:03,449 INFO      DRY-RUN: delete_indices: cloudtrail-dev-2017.09.08 (CLOSED) with arguments: {}
2017-10-02 18:56:03,449 INFO      DRY-RUN: delete_indices: cloudtrail-dev-2017.09.09 (CLOSED) with arguments: {}
2017-10-02 18:56:03,449 INFO      DRY-RUN: delete_indices: cloudtrail-dev-2017.09.10 (CLOSED) with arguments: {}
2017-10-02 18:56:03,449 INFO      DRY-RUN: delete_indices: cloudtrail-dev-2017.09.11 (CLOSED) with arguments: {}
2017-10-02 18:56:03,449 INFO      DRY-RUN: delete_indices: cloudtrail-dev-2017.09.12 (CLOSED) with arguments: {}
2017-10-02 18:56:03,449 INFO      DRY-RUN: delete_indices: cloudtrail-dev-2017.09.13 (CLOSED) with arguments: {}
2017-10-02 18:56:03,449 INFO      DRY-RUN: delete_indices: cloudtrail-dev-2017.09.14 (CLOSED) with arguments: {}
2017-10-02 18:56:03,449 INFO      DRY-RUN: delete_indices: cloudtrail-dev-2017.09.15 (CLOSED) with arguments: {}
2017-10-02 18:56:03,449 INFO      DRY-RUN: delete_indices: cloudtrail-dev-2017.09.16 (CLOSED) with arguments: {}
2017-10-02 18:56:03,449 INFO      DRY-RUN: delete_indices: cloudtrail-dev-2017.09.17 with arguments: {}
2017-10-02 18:56:03,449 INFO      DRY-RUN: delete_indices: cloudtrail-dev-2017.09.18 with arguments: {}
2017-10-02 18:56:03,452 INFO      Action ID: 1, "delete_indices" completed.
2017-10-02 18:56:03,452 INFO      Job completed.

--
Niraj

4

You have 2 filters blocks. That will not work. You must remove one or merge them.

5

@theuntergeek Thanks. Seems like removing the last one worked perfectly fine. Appreciate the help.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.