Hello, I'm using ElasticSearch v6.3.2 and Curator v5.7.6 to delete indices. Unfortunately it seems that my configuration doesn't find any indices even though there are many.
My commands:
$ curator_cli --host elasticsearch show_indices --verbose
.kibana open 54.9KB 8 1 1 2019-04-08T14:45:06Z
logstash-2019.06.01 open 7.6GB 15639686 5 1 2019-06-01T00:00:02Z
logstash-2019.06.02 open 4.9GB 8438820 5 1 2019-06-02T00:00:01Z
logstash-2019.06.03 open 9.0GB 19123676 5 1 2019-06-03T00:00:00Z
logstash-2019.06.04 open 10.6GB 23192736 5 1 2019-06-04T00:00:01Z
logstash-2019.06.05 open 10.1GB 21986756 5 1 2019-06-05T00:00:01Z
logstash-2019.06.06 open 10.7GB 23218430 5 1 2019-06-06T00:00:01Z
[...]
$ curator_cli --host elasticsearch show_indices --filter_list '{"filtertype":"age","source":"name","timestring":"%Y.%m.%d","unit":"days","unit_count":30,"direction":"older"}'
2019-06-26 13:15:30,201 ERROR Singleton action failed due to empty index list
I've read already two posts with the same error but it seems we don't have the same issue:
I already executed the command to display the indices, as I displayed.
Here are the files, config.yml:
# Remember, leave a key empty if there is no value. None will be a string,
# not a Python "NoneType"
client:
hosts:
- elasticsearch
port: 9200
url_prefix:
use_ssl: False
certificate:
client_cert:
client_key:
ssl_no_validate: False
http_auth:
timeout: 30
master_only: False
logging:
loglevel: INFO
logfile:
logformat: default
blacklist: ['elasticsearch', 'urllib3']
action_file.yml:
# Remember, leave a key empty if there is no value. None will be a string,
# not a Python "NoneType"
#
# Also remember that all examples have 'disable_action' set to True. If you
# want to use this action as a template, be sure to set this to False after
# copying it.
actions:
1:
action: delete_indices
description: "Clean up ES by deleting indices older than 30 days"
options:
timeout_override:
continue_if_exception: False
disable_action: False
ignore_empty_list: True
filters:
- filtertype: age
source: name
direction: older
timestring: '%Y.%m.%d'
unit: days
unit_count: 30
field:
stats_result:
epoch:
exclude: False
This doesn't indicate no indices. It indicates that your filter set resulted in no actionable indices. Running with --loglevel DEBUG should show more information about how Curator reached this conclusion.
I've tried, but the result of dry-run seems to still not find anything:
2019-06-26 14:49:55,690 INFO Preparing Action ID: 1, "delete_indices"
2019-06-26 14:49:55,707 INFO Trying Action ID: 1, "delete_indices": Clean up ES by deleting indices older than 30 days
2019-06-26 14:49:56,198 INFO DRY-RUN MODE. No changes will be made.
2019-06-26 14:49:56,199 INFO (CLOSED) indices may be shown that may not be acted on by action "delete_indices".
2019-06-26 14:49:56,199 INFO Action ID: 1, "delete_indices" completed.
2019-06-26 14:49:56,199 INFO Job completed.
I have two ElasticSearch cluster, they are the same, one is used as production, the other one as sandbox. I used the exact same config and action files as mentioned, and it worked fine on my sandbox.
Another thing, I noticed that my indices are not all green, but some that are supposed to be deleted are green so I don't think that is linked.
building preliminary index metadata for every indices (OK)
generating working list of indices (OK)
it applies filter (KO 'cause doesn't match my indices but it should)
show results
Here are some logs:
2019-06-26 15:02:37,000 DEBUG curator.indexlist filter_by_age:489 Filtering indices by age
2019-06-26 15:02:37,000 DEBUG curator.indexlist _get_name_based_ages:280 Getting ages of indices by "name"
2019-06-26 15:02:37,000 DEBUG curator.indexlist empty_list_check:226 Checking for empty list
2019-06-26 15:02:37,000 DEBUG curator.utils get_date_regex:200 regex = \d{4}\.\d{2}\.\d{2}
2019-06-26 15:02:37,000 DEBUG curator.indexlist working_list:237 Generating working list of indices
2019-06-26 15:02:37,004 DEBUG curator.indexlist working_list:237 Generating working list of indices
2019-06-26 15:02:37,004 DEBUG curator.indexlist __not_actionable:39 Index logstash-2019.06.03 is not actionable, removing from list.
2019-06-26 15:02:37,004 DEBUG curator.indexlist __excludify:58 Removed from actionable list: Index "logstash-2019.06.03" age (1559520000), direction: "older", point of reference, (1558969357)
2019-06-26 15:02:37,005 DEBUG curator.indexlist __not_actionable:39 Index logstash-2019.06.24 is not actionable, removing from list.
2019-06-26 15:02:37,005 DEBUG curator.indexlist __excludify:58 Removed from actionable list: Index "logstash-2019.06.24" age (1561334400), direction: "older", point of reference, (1558969357)
[...]
2019-06-26 15:02:37,099 DEBUG curator.indexlist iterate_filters:1242 Post-instance: []
2019-06-26 15:02:37,100 DEBUG curator.indexlist empty_list_check:226 Checking for empty list
2019-06-26 15:02:37,100 ERROR curator.cli_singletons.cli_action.show_indices do_filters:142 Singleton action failed due to empty index list
point of reference is Monday, May 27, 2019 3:02:37 PM
The point of reference is determined by unit: days and unit_count: 30, which is merely the value of unit_count multiplied by the number of seconds in unit, and then subtracted (this is epoch time) from the time at Curator execution.
Hence, logstash-2019.06.03 is not older than your point of reference, so it is removed from the actionable list. You can verify all of the indices in your cluster using these epoch times similarly.
Hm, I realized something strange, I'm checking with Kibana the indices present in ElasticSearch, but the command curator_cli --host elasticsearch show_indices doesn't show me the same results... I don't understand that
This is unusual. It implies that the Elasticsearch cluster Kibana is pointing to differs from the one curator_cli is pointing to, or that you have a split-brain scenario in your cluster (subsections think they are the whole cluster).
I came back and it appears that the cluster health was not good, I dunno exactly why but it turned all green during the weekend. In the morning the indices returned by the curator_cli were the same as on Kibana, and my initial configuration is working well now.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.