Elastic search curator is failing

Elastic Stack Elasticsearch
1

[root@BVELKL03 elasticsearch-curator]# curator_cli --host localhost show_indices --filter_list '{"filtertype":"age","source":"name","timestring":"%Y.%m.%d","unit":"days","unit_count":30}'
2018-06-26 07:07:23,177 ERROR Schema error: required key not provided @ data['direction']
2018-06-26 07:07:23,177 ERROR Schema error: Configuration: filter: Location: singleton, filter #0: {'filtertype': 'age', 'source': 'name', 'timestring': '%Y.%m.%d', 'unit': 'days', 'unit_count': 30}: Bad Value: "(could not determine)", required key not provided @ data['direction']. Check configuration file.
2018-06-26 07:07:23,177 CRITICAL Unable to parse filters: Configuration: filters: Location: show_indices singleton action "filters": Bad Value: "None", Configuration: filter: Location: singleton, filter #0: {'filtertype': 'age', 'source': 'name', 'timestring': '%Y.%m.%d', 'unit': 'days', 'unit_count': 30}: Bad Value: "(could not determine)", required key not provided @ data['direction']. Check configuration file.. Check configuration file.

I have the following error couldn't understand where to find of save the configuration file

2

Hey i was able to fix this issue, but how do i make sure my indices are deleted
root@BVELKL03 elasticsearch-curator]# curator_cli --config curator.yml show_indices --filter_list '{"filtertype":"age","source":"name","timestring":"%Y.%m.%d","unit":"days","unit_count":30}'

this is returning empty does this mean that the indices are deleted

3

If you run curator_cli --config curator.yml show_indices without a --filter_list, it will show you all indices in the cluster.

4

@theuntergeek can you provide a sample to delete the indices which are not time based
actions:
1:
action: delete_snapshots
description: >-
Delete indices older than 30 days (based on index name), for logstash-
prefixed indices. Ignore the error if the filter does not result in an
actionable list of indices (ignore_empty_list) and exit cleanly.
options:
ignore_empty_list: True
timeout_override:
continue_if_exception: False
disable_action: False
filters:
- filtertype: age
source: field_stats
direction: older
unit: hours
unit_count: 1

this is not working I have timebased indices and non time based indices

5

To use source: field_stats you need to provide a field name (if not @timestamp), as well as stats_result if not using min_value as the value for stats_result.

How will you determine which indices to delete? Index names alone? Will you need to exclude time-series indices?

6

I have problem i want to delete indices irrespective of whether time based or non time based indices which are older than 7 days

7

Your recourse is to use source: creation_date, which is set at index creation time:

- filtertype: age
  source: creation_date
  direction: older
  unit: days
  unit_count: 7

This will delete indices older than 7 days based on when they were created, with no regard to any other criteria but the index creation date.

8

even after deleting i can data in kibana

9

i meant to say al the indices which are older are available in kibana

10

actions:
1:
action: delete_indices
description: >-
Delete indices older than 5 minutes.
options:
ignore_empty_list: True
timeout_override:
continue_if_exception: False
disable_action: False
filters:
- filtertype: age
source: creation_date
direction: older
unit: days
unit_count: 7

unable to delete using the following actions2.yml

curator.yml

Remember, leave a key empty if there is no value. None will be a string,

not a Python "NoneType"

client:
hosts:

  • 127.0.0.1
    port: 9200
    url_prefix:
    use_ssl: False
    certificate:
    client_cert:
    client_key:
    ssl_no_validate: False
    http_auth:
    timeout: 30
    master_only: False

logging:
loglevel: INFO
logfile:
logformat: default
blacklist: []

11

YAML is indent-sensitive. I can't troubleshoot your configuration without the original formatting preserved. Please use the </> code tags or encapsulate your pasted content inside of triple back ticks, like this:

```
PASTE HERE
```

Were there any logs resulting from your attempt?

12

thanks untergreek i validated using yaml validator it resolved

curator_cli --config curator.yml delete_indices --filter_list '{"filtertype":"age","source":"creation_date","direction":"older","unit":"days","unit_count":7}'

I am planning to use this command to delete the indices every in cron and I have an usecase where I want to delete few index patterns sesvg1,sppvg1,sppvg2 every 30 days and few index patterns named tmoes, sem every 7 days can you help on how to achieve using the same command

13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.