Yesterday I received help here that allowed me to get curator to run. I tested my action.yml in dry-run mode yesterday - and all looked fine.
I am brand new to ELK and Curator - so please bear with me
This morning - I went to run the action for real - and got an error.
Here is my action.yml:
# cat /etc/curator/action.yml
actions:
-
1:*
-
action: close*
-
description: >-*
-
Close indices older than 30 days (based on creation date).* -
options:*
-
ignore_empty_list: True* -
delete_aliases: False* -
disable_action: False* -
filters:*
-
- filtertype: age*
-
source: creation_date* -
direction: older* -
timestring: '%Y.%m.%d'* -
unit: days* -
unit_count: 30* -
2:*
-
action: delete_indices*
-
description: >-*
-
Delete indices older than 30 days (based on creation date).* -
options:*
-
ignore_empty_list: True* -
disable_action: False* -
filters:*
-
- filtertype: age*
-
source: creation_date* -
direction: older* -
timestring: '%Y.%m.%d'* -
unit: days* -
unit_count: 30*
When I run in dry-run mode - I get job completed. When I run 'for real' I pulled this error from the log (debug). This is the last two lines of the log #724 and #725:
*724 2020-03-26 12:18:48,053 INFO curator.actions.close do_action:305 Closing 15 selected indices: ['aapc.log-stage', 'aapc.log-prod', 'aapc.log-local', '.kibana6_ 8', '.kibana6_7', 'searchguard', '.kibana_task_manager', 'shipapidev-', 'wwwdev-', 'aapc.log-test', '.tasks', 'aapc.log-navigators', '.kibana6_9', '.security', '.kibana']*
- 725 2020-03-26 12:18:48,981 ERROR curator.cli run:213 Failed to complete action: close. <class 'curator.exceptions.FailedExecution'>: Exception en countered. Rerun with loglevel DEBUG and/or check Elasticsearch logs for more information. Exception: ConflictError(409, '{"_shards":{"total":79,"successful":41,"failed":2},"aap c.log-prod":{"total":10,"successful":3,"failed":2,"failures":[{"shard":0,"reason":"pending operations","routing":{"state":"STARTED","primary":true,"node":"hNGMw-wTSZyXMuMJoJG4QQ" ,"relocating_node":null,"shard":0,"index":"aapc.log-prod","allocation_id":{"id":"IWU3OJVsTPq4rh04uvZCQg"}}},{"shard":1,"reason":"pending operations","routing":{"state":"STARTED", "primary":true,"node":"hNGMw-wTSZyXMuMJoJG4QQ","relocating_node":null,"shard":1,"index":"aapc.log-prod","allocation_id":{"id":"NzFW9d9tTPCF6CsJuU3bwQ"}}}]},".kibana_task_manager" :{"total":1,"successful":1,"failed":0},"aapc.log-test":{"total":10,"successful":5,"failed":0},"shipapidev-":{"total":10,"successful":5,"failed":0},".tasks":{"total":1,"successful ":1,"failed":0},"aapc.log-navigators":{"total":10,"successful":5,"failed":0},".security":{"total":1,"successful":1,"failed":0},"searchguard":{"total":1,"successful":1,"failed":0} ,".kibana":{"total":2,"successful":1,"failed":0},"aapc.log-stage":{"total":10,"successful":5,"failed":0},".kibana6_9":{"total":1,"successful":1,"failed":0},"aapc.log-local":{"tot al":10,"successful":5,"failed":0},".kibana6_8":{"total":1,"successful":1,"failed":0},".kibana6_7":{"total":1,"successful":1,"failed":0},"wwwdev-":{"total":10,"successful":5,"fail ed":0}}')*
If I read this right, the error is during the close section of action.yml. It never gets to the delete at all.