Elasticsearch Curator error

Elastic Stack Elasticsearch
1

Hi all,
I am using ELK curator to delete old elasticsearch indices periodically. I am running this as kubernetes cronjob to delete older indices.
I am facing the following error-
2020-02-18 15:54:51,628 ERROR curator.cli run:213 Failed to complete action: delete_indices. <class 'KeyError'>: 'indices'

Following are my configurations -

actions:
  1:
    action: delete_indices
    description: >-
      Delete indices older than 2 weeks (based on index name), for vae
      prefixed indices. Ignore the error if the filter does not result in an
      actionable list of indices (ignore_empty_list) and exit cleanly.
    options:
      ignore_empty_list: True
      timeout_override:
      continue_if_exception: False
      disable_action:
    filters:
    - filtertype: pattern
      kind: regex
      value: '^prod-geo-localhostaccesslog'
      exclude:
    - filtertype: age
      source: creation_date
      direction: older
      timestring: '%Y.%W'
      unit: weeks
      unit_count: 1


  config.yml: |-
    client:
      hosts:
        - https://es.test.com
      port:
      url_prefix:
      use_ssl: False
      certificate:
      client_cert:
      client_key:
      aws_key:
      aws_secret_key:
      aws_region:
      ssl_no_validate: False
      http_auth:
      timeout: 30
      master_only: False

    logging:
      loglevel: INFO
      logfile:
      logformat: default
      blacklist: ['elasticsearch', 'urllib3']
2

Hi Rajat,
Are you indices created on weekly basis or daily basis ? In description you say delete indices older than 2 weeks, but count is 1 (considering unit weeks, it will delete indices older than a week), may be you need to relook here.
Also, try with this config:

filters: 
  - 
    exclude: ~
    filtertype: pattern
    kind: prefix
    value: prod-geo-localhostaccesslog
  - 
    direction: older
	exclude: ~
    filtertype: age
	source: creation_date
	timestring: "%Y.%W"
	unit: weeks
	unit_count: 1
options: 
  continue_if_exception: false
  disable_action: false
  ignore_empty_list: true
  timeout_override: ~
3

@Ayush_Mathur sorry, forgot to change the description. Thanks for the config i ll try this out.

4

@Ayush_Mathur is the syntax correct? the hyphens and all, although i tried with the following but same error.

options:
      ignore_empty_list: True
      timeout_override: ~
      continue_if_exception: False
      disable_action: false
    filters:
    - filtertype: pattern
      kind: prefix
      value: prod-geo-localhostaccesslog
      exclude:
    - filtertype: age
      source: creation_date
      direction: older
      timestring: '%Y.%W'
      unit: weeks
      unit_count: 1
5

@rj23495 agree the indentation is bit off, but assumed you would fix it.
Anyway, are your indices name starting with prod-geo-localhostaccesslog ? If yes, this config should work (using the same approach in my project).

6

yes, my index starts with prod-geo-localhostaccesslog-2020.02 etc something like this.
I tried with the configs u sent but still same error.
Also ES version is 6.2.3
curator version - 5.8.1

7

OK, can you check if you are able to curl your ES host (https://es.test.com) from inside your curator container ?
My guess is you're missing either authentication params or certificates/ ssl_no_validate should be True/ host should be changed to logging-es with port 9200 (or whatever cluster name you're using).

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.