Unable to delete indices using curator

Elastic Stack Elasticsearch
1

Hi , Using curator but not performing any action . Can someone help

    [root@iacapps-es-node1 elasticsearch-curator]# curator --config curator.yml delete_indices.yml
    2018-02-18 00:10:01,892 DEBUG                curator.cli                    run:108  Client and logging options validated.
    2018-02-18 00:10:01,892 DEBUG                curator.cli                    run:112  default_timeout = 30
    2018-02-18 00:10:01,892 DEBUG                curator.cli                    run:116  action_file: delete_indices.yml
    2018-02-18 00:10:01,896 DEBUG                curator.cli                    run:118  action_config: {'actions': {1: {'action': 'delete_indices', 'description': 'Delete indices older than X days (based on index name), for logstash- prefixed indices.', 'options': {'ignore_empty_list': False, 'disable_action': False}, 'filters': [{'filtertype': 'pattern', 'kind': 'prefix', 'value': 'logs-download-', 'exclude': None}, {'filtertype': 'age', 'source': 'name', 'direction': 'older', 'timestring': '%Y.%m.%d', 'unit': 'days', 'unit_count': 2}]}}}
    2018-02-18 00:10:01,896 DEBUG     curator.validators.SchemaCheck               __init__:26   Schema: {'actions': <class 'dict'>}
    2018-02-18 00:10:01,896 DEBUG     curator.validators.SchemaCheck               __init__:27   "Actions File" config: {'actions': {1: {'action': 'delete_indices', 'description': 'Delete indices older than X days (based on index name), for logstash- prefixed indices.', 'options': {'ignore_empty_list': False, 'disable_action': False}, 'filters': [{'filtertype': 'pattern', 'kind': 'prefix', 'value': 'logs-download-', 'exclude': None}, {'filtertype': 'age', 'source': 'name', 'direction': 'older', 'timestring': '%Y.%m.%d', 'unit': 'days', 'unit_count': 2}]}}}
    2018-02-18 00:10:01,897 DEBUG     curator.validators.SchemaCheck               __init__:26   Schema: {'action': Any([In(['alias', 'allocation', 'close', 'cluster_routing', 'create_index', 'delete_indices', 'delete_snapshots', 'forcemerge', 'index_settings', 'open', 'reindex', 'replicas', 'restore', 'rollover', 'shrink', 'snapshot'])])}
    2018-02-18 00:10:01,897 DEBUG     curator.validators.SchemaCheck               __init__:27   "action type" config: {'action': 'delete_indices', 'description': 'Delete indices older than X days (based on index name), for logstash- prefixed indices.', 'options': {'ignore_empty_list': False, 'disable_action': False}, 'filters': [{'filtertype': 'pattern', 'kind': 'prefix', 'value': 'logs-download-', 'exclude': None}, {'filtertype': 'age', 'source': 'name', 'direction': 'older', 'timestring': '%Y.%m.%d', 'unit': 'days', 'unit_count': 2}]}
    2018-02-18 00:10:01,902 DEBUG     curator.validators.SchemaCheck               __init__:26   Schema: <function Filters.<locals>.f at 0x7f9ced0a0598>
    2018-02-18 00:10:01,902 DEBUG     curator.validators.SchemaCheck               __init__:27   "filters" config: [{'filtertype': 'pattern', 'kind': 'prefix', 'value': 'logs-download-', 'exclude': None}, {'filtertype': 'age', 'source': 'name', 'direction': 'older', 'timestring': '%Y.%m.%d', 'unit': 'days', 'unit_count': 2}]
    2018-02-18 00:10:01,902 DEBUG     curator.validators.SchemaCheck               __init__:26   Schema: {'filtertype': Any([In(['age', 'alias', 'allocated', 'closed', 'count', 'forcemerged', 'kibana', 'none', 'opened', 'pattern', 'period', 'space', 'state'])]), 'kind': Any(['prefix', 'suffix', 'timestring', 'regex']), 'value': Any([<class 'str'>, <class 'str'>]), 'exclude': Any([<class 'bool'>, All(Any([<class 'str'>, <class 'str'>]), <function Boolean at 0x7f9ced055730>, msg=None)])}
    2018-02-18 00:10:01,902 DEBUG     curator.validators.SchemaCheck               __init__:27   "filter" config: {'filtertype': 'pattern', 'kind': 'prefix', 'value': 'logs-download-'}
    2018-02-18 00:10:01,902 DEBUG     curator.validators.filters                      f:48   Filter #0: {'filtertype': 'pattern', 'kind': 'prefix', 'value': 'logs-download-', 'exclude': False}
    2018-02-18 00:10:01,903 DEBUG     curator.defaults.filtertypes                    age:56   AGE FILTER = [{'direction': Any(['older', 'younger'])}, {'unit': Any(['seconds', 'minutes', 'hours', 'days', 'weeks', 'months', 'years'])}, {'unit_count': Coerce(int, msg=None)}, {'unit_count_pattern': Any([<class 'str'>, <class 'str'>])}, {'epoch': Any([Coerce(int, msg=None), None])}, {'exclude': Any([<class 'bool'>, All(Any([<class 'str'>, <class 'str'>]), <function Boolean at 0x7f9ced0787b8>, msg=None)])}, {'source': Any(['name', 'creation_date', 'field_stats'])}, {'stats_result': Any(['min_value', 'max_value'])}, {'timestring': Any([<class 'str'>, <class 'str'>])}]
    2018-02-18 00:10:01,903 DEBUG     curator.validators.SchemaCheck               __init__:26   Schema: {'filtertype': Any([In(['age', 'alias', 'allocated', 'closed', 'count', 'forcemerged', 'kibana', 'none', 'opened', 'pattern', 'period', 'space', 'state'])]), 'direction': Any(['older', 'younger']), 'unit': Any(['seconds', 'minutes', 'hours', 'days', 'weeks', 'months', 'years']), 'unit_count': Coerce(int, msg=None), 'unit_count_pattern': Any([<class 'str'>, <class 'str'>]), 'epoch': Any([Coerce(int, msg=None), None]), 'exclude': Any([<class 'bool'>, All(Any([<class 'str'>, <class 'str'>]), <function Boolean at 0x7f9ced0787b8>, msg=None)]), 'source': Any(['name', 'creation_date', 'field_stats']), 'stats_result': Any(['min_value', 'max_value']), 'timestring': Any([<class 'str'>, <class 'str'>])}
    2018-02-18 00:10:01,903 DEBUG     curator.validators.SchemaCheck               __init__:27   "filter" config: {'filtertype': 'age', 'source': 'name', 'direction': 'older', 'timestring': '%Y.%m.%d', 'unit': 'days', 'unit_count': 2}
    22018-02-18 00:10:01,904 DEBUG                curator.cli                    run:126  action_disabled = False
    2018-02-18 00:10:01,904 DEBUG                curator.cli                    run:130  continue_if_exception = False
    2018-02-18 00:10:01,904 DEBUG                curator.cli                    run:132  timeout_override = None
    2018-02-18 00:10:01,904 DEBUG                curator.cli                    run:134  ignore_empty_list = False
    2018-02-18 00:10:01,904 INFO                 curator.cli                    run:144  Preparing Action ID: 1, "delete_indices"
    2018-02-18 00:10:01,904 DEBUG              curator.utils             get_client:803  kwargs = {'hosts': ['10.62.4.15'], 'port': 9200, 'use_ssl': False, 'certificate': '/etc/elasticsearch/certs/ca.crt', 'ssl_no_validate': False, 'http_auth': 'elastic:Ducati', 'master_only': False, 'client_key': None, 'aws_key': None, 'aws_sign_request': False, 'client_cert': None, 'aws_token': None, 'url_prefix': '', 'aws_secret_key': None, 'timeout': 30}
    2018-02-18 00:10:01,907 DEBUG              curator.utils             get_client:880  Not using "requests_aws4auth" python module to connect.
2

Please format your code, logs or configuration files using </> icon as explained in this guide and not the citation button. It will make your post more readable.

Or use markdown style like:

```
CODE
```

There's a live preview panel for exactly this reasons.

Lots of people read these forums, and many of them will simply skip over a post that is difficult to read, because it's just too large an investment of their time to try and follow a wall of badly formatted text.
If your goal is to get an answer to your questions, it's in your interest to make it as easy to read and understand as possible.
Please update your post.

3

Thanks . I have updated the format as per your advise

4

Thanks. Can you share your curator configuration file?

5

curator.yml is below also I am using x-pack generated certs for authentication . I tried with and without cert but didnt helped

    ---
    # Remember, leave a key empty if there is no value.  None will be a string,
    # not a Python "NoneType"
    client:
      hosts:
        - 10.62.4.15
      port: 9200
      url_prefix:
      use_ssl: False
      certificate: '/etc/elasticsearch/certs/ca.crt'
#      certificate:
      client_cert:
      client_key:
      ssl_no_validate: False
      http_auth: elastic:Ducati1199
      timeout: 30
      master_only: False

    logging:
      loglevel: DEBUG
      logfile:
      logformat: default
      blacklist: ['elasticsearch', 'urllib3']
6

This is only how it connects to elasticsearch. you need to specify actions to execute..

Have a look here: https://www.elastic.co/guide/en/elasticsearch/client/curator/current/examples.html

and here: https://www.elastic.co/guide/en/elasticsearch/client/curator/current/ex_delete_indices.html

7

Pjanzen ,

I do have action file ..if you checked the previous comments .

---
# Remember, leave a key empty if there is no value.  None will be a string,
# not a Python "NoneType"
#
# Also remember that all examples have 'disable_action' set to True.  If you
# want to use this action as a template, be sure to set this to False after
# copying it.
actions:
  1:
    action: delete_indices
    description: >-
      Delete indices older than X days (based on index name), for logstash-
      prefixed indices.
    options:
      ignore_empty_list: False
      disable_action: False
    filters:
    - filtertype: pattern
      kind: prefix
      value: logs-download-
    - filtertype: age
      source: name
      direction: older
      timestring: '%Y.%m.%d'
      unit: days
      unit_count: 2
8

can someone advise on this ?

9

Change this to:

blacklist: []

And run again. There should be connection data in the log file that will reveal where the connection is having issues.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.