Curator action, delete_indices problem

Elastic Stack Elasticsearch
1

Hello guys,

I have ELK reading a folder with CSV's, everything is working like a charm. I installed curator to delete indices older than 7 days, but I'm in a testing environment so I used minutes instead of days. I've started Logstash at 18:05 GMT to start inserting in Elastic, then at 18:30 GMT I did a echo to a CSV to have at least one row with a controlable timegap.

My config file is has follow, runned at 18:31 GMT:

actions:
1:
action: delete_indices
description: >-
Delete indices older than 45 days (based on index name), for logstash-
prefixed indices. Ignore the error if the filter does not result in an
actionable list of indices (ignore_empty_list) and exit cleanly.
options:
ignore_empty_list: True
timeout_override:
continue_if_exception: False
disable_action: False
filters:
- filtertype: pattern
kind: prefix
value: logstash_
exclude:
- filtertype: age
source: creation_date
direction: older
unit: minutes
unit_count: 20
exclude:

But curator delete all index not the ones older than 20 minutes, so should ignore the entry at 18:30. Curator output:

2016-07-06 18:31:55,828 INFO Action #1: delete_indices
2016-07-06 18:31:55,829 INFO Starting new HTTP connection (1): 127.0.0.1
2016-07-06 18:31:55,834 INFO GET http://127.0.0.1:9200/ [status:200 request:0.005s]
2016-07-06 18:31:55,836 INFO GET http://127.0.0.1:9200/_all/_settings?expand_wildcards=open%2Cclosed [status:200 request:0.001s]
2016-07-06 18:31:55,838 INFO GET http://127.0.0.1:9200/_cluster/state/metadata/.kibana,logstash_2016-07-06 [status:200 request:0.001s]
2016-07-06 18:31:55,842 INFO GET http://127.0.0.1:9200/.kibana,logstash_2016-07-06/_stats/store,docs [status:200 request:0.004s]
2016-07-06 18:31:55,842 INFO Index .kibana is not actionable, removing from list.
2016-07-06 18:31:55,843 INFO Index logstash_2016-07-06 is actionable and remains in the list.
2016-07-06 18:31:55,843 INFO Index logstash_2016-07-06 is actionable and remains in the list.
2016-07-06 18:31:55,843 INFO Remains in actionable list: Index "logstash_2016-07-06" age (1467824921), direction: "older", point of reference, (1467825115)
2016-07-06 18:31:55,843 INFO Deleting selected indices
2016-07-06 18:31:55,844 INFO ---deleting index logstash_2016-07-06
2016-07-06 18:31:55,880 INFO DELETE http://127.0.0.1:9200/logstash_2016-07-06?master_timeout=30s [status:200 request:0.036s]
2016-07-06 18:31:55,881 INFO GET http://127.0.0.1:9200/_all/_settings?expand_wildcards=open%2Cclosed [status:200 request:0.001s]

Can anyone help me? What I'm doing wrong?

Thanks in advanced :slight_smile:

2

I only see one matching index, logstash_2016-07-06. Your description seems inaccurate:

Delete indices older than 45 days (based on index name), for logstash-
prefixed indices. Ignore the error if the filter does not result in an
actionable list of indices (ignore_empty_list) and exit cleanly.

What I see is that you're filtering based on prefix (logstash_, and that's clearly working), and by age. The age your configuration suggests is a creation_date older than 20 minutes ago. The creation date of the logstash_2016-07-06 is more than 20 minutes ago, so it remains in the actionable list.

I don't see anything wrong here. It's doing exactly what you told it to do. If this is not what you want Curator to do, please advise and I will see if I can help you reconfigure.

3

For further consideration:

2016-07-06 18:31:55,843 INFO Remains in actionable list: Index "logstash_2016-07-06" age (1467824921), direction: "older", point of reference, (1467825115)

The creation_date is 1467824921, which is Wed, 06 Jul 2016 17:08:41 GMT
The point of reference is 1467825115, which is Wed, 06 Jul 2016 17:11:55 GMT, which is 20 minutes before 17:31:55 GMT. You're apparently ahead of GMT by 1 hour, so your log entry reads 2016-07-06 18:31:55,843

The raw comparison states that the index was created at 17:08:41, which is older than your designated reference point of 17:11:55, so the index is slated for deletion.

4

Hello Aaron,

That would be a great point, and actually you were correct. To avoid a smaller timespace, I did other test.

Yesterday I left the data from Elastic, added new data about 5 minutes ago. And reconfigured action file to:
unit: hours unit_count: 12

So it should delete indices older than 12 hours, weirdly deleted all.

5

Hello again Aaron,

I did it with minutes search, had 20K records each 10k block with difference of 15 min. Runned action file with unit in minutes older than 15min and it worked :). Was your tip of the timestamp :slight_smile:

Thanks for your support.

Best regards,
Pedro Lopes