Curator date match does not meet provided criteria

Elastic Stack Elasticsearch
1

I'm trying to run curator and getting "does not meet provided criteria. Removing from list." Any idea on why the pattern below is not matching the index name?

  3:
    action: delete_indices
    description: >-
      Delete stats-v2-events-session-new indices older than 5 days (based on index name), for logstash-
      prefixed indices. Ignore the error if the filter does not result in an
      actionable list of indices (ignore_empty_list) and exit cleanly.
    options:
      ignore_empty_list: True
      timeout_override:
      continue_if_exception: False
      disable_action: False
    filters:
    - filtertype: pattern
      kind: prefix
      value: 'stats-v2-events-session-new-vhb-'
      exclude:
    - filtertype: age
      source: name
      direction: older
      timestring: '%d.%m.%Y '
      unit: days
      unit_count: 3
      exclude:

2021-01-27 00:51:19,427 DEBUG          curator.indexlist          filter_by_age:490  Filtering indices by age
2021-01-27 00:51:19,427 DEBUG          curator.indexlist   _get_name_based_ages:280  Getting ages of indices by "name"
2021-01-27 00:51:19,427 DEBUG          curator.indexlist       empty_list_check:226  Checking for empty list
2021-01-27 00:51:19,427 DEBUG              curator.utils         get_date_regex:195  Provided timestring = "%d.%m.%Y "
2021-01-27 00:51:19,427 DEBUG              curator.utils         get_date_regex:197  Current character: % Array position: 0
2021-01-27 00:51:19,427 DEBUG              curator.utils         get_date_regex:197  Current character: d Array position: 1
2021-01-27 00:51:19,427 DEBUG              curator.utils         get_date_regex:197  Current character: . Array position: 2
2021-01-27 00:51:19,427 DEBUG              curator.utils         get_date_regex:197  Current character: % Array position: 3
2021-01-27 00:51:19,427 DEBUG              curator.utils         get_date_regex:197  Current character: m Array position: 4
2021-01-27 00:51:19,427 DEBUG              curator.utils         get_date_regex:197  Current character: . Array position: 5
2021-01-27 00:51:19,427 DEBUG              curator.utils         get_date_regex:197  Current character: % Array position: 6
2021-01-27 00:51:19,427 DEBUG              curator.utils         get_date_regex:197  Current character: Y Array position: 7
2021-01-27 00:51:19,427 DEBUG              curator.utils         get_date_regex:197  Current character:   Array position: 8
2021-01-27 00:51:19,427 DEBUG              curator.utils         get_date_regex:207  regex = \d{2}\.\d{2}\.\d{4} 
2021-01-27 00:51:19,427 DEBUG          curator.indexlist           working_list:237  Generating working list of indices
2021-01-27 00:51:19,428 DEBUG          curator.indexlist           working_list:237  Generating working list of indices
2021-01-27 00:51:19,428 DEBUG          curator.indexlist          filter_by_age:550  Index "stats-v2-events-session-new-vhb-23.1.2021" does not meet provided criteria. Removing from list.
2021-01-27 00:51:19,428 DEBUG          curator.indexlist          filter_by_age:550  Index "stats-v2-events-session-new-vhb-22.1.2021" does not meet provided criteria. Removing from list.
2021-01-27 00:51:19,428 DEBUG          curator.indexlist          filter_by_age:550  Index "stats-v2-events-session-new-vhb-21.1.2021" does not meet provided criteria. Removing from list.
2021-01-27 00:51:19,428 DEBUG          curator.indexlist          filter_by_age:550  Index "stats-v2-events-session-new-vhb-25.1.2021" does not meet provided criteria. Removing from list.
2021-01-27 00:51:19,428 DEBUG          curator.indexlist          filter_by_age:550  Index "stats-v2-events-session-new-vhb-19.1.2021" does not meet provided criteria. Removing from list.
2021-01-27 00:51:19,428 DEBUG          curator.indexlist          filter_by_age:550  Index "stats-v2-events-session-new-vhb-27.1.2021" does not meet provided criteria. Removing from list.
2021-01-27 00:51:19,428 DEBUG          curator.indexlist          filter_by_age:550  Index "stats-v2-events-session-new-vhb-20.1.2021" does not meet provided criteria. Removing from list.
2021-01-27 00:51:19,428 DEBUG          curator.indexlist          filter_by_age:550  Index "stats-v2-events-session-new-vhb-26.1.2021" does not meet provided criteria. Removing from list.
2

Please see the warning, A word about regular expression matching with timestrings

Your month is not zero padded, and I would expect that single-digit days to also not be zero padded. This will result in the date not being recognized.

Unless the data inside does not correspond with the index creation date, I recommend using creation_date as your age filter source, rather than name. It removes this ambiguity by using the actual date stamp when the index was created. If you must use name, you'll have to fix your index naming algorithm to provide zero-padded days and months.

3

thank you, that worked.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.