Index delete using curator 5.1

Elastic Stack Elasticsearch
1

Hi ,
I'm using curator to delete my index by time basic, but in my index name doesn't contain timestring.

How could i perform this delete operation.

ACTION.YMl

---
# Remember, leave a key empty if there is no value.  None will be a string,
# not a Python "NoneType"
#
# Also remember that all examples have 'disable_action' set to True.  If you
# want to use this action as a template, be sure to set this to False after
# copying it.
actions:
  1:
    action: delete_indices
    description: delete
    options:
      ignore_empty_list: True
      disable_action: false
      continue_if_exception : True
    filters:
    #- filtertype:kibana
    #  exclude:False   
    - filtertype: age
      source: field_stats
      field: 'log_timestamp'        
      direction: older
      unit: days
      unit_count: 90

Index name be like this "logstash-dd.log"

2

Does this action file not work? It seems properly configured. Try running with --dry-run to see the list that would be deleted on an actual run.

3

When i run the above file i'm getting following error. Its searching different index and throwing error.

2017-06-22 23:00:59,742 INFO      Preparing Action ID: 1, "delete_indices"
2017-06-22 23:00:59,750 INFO      Trying Action ID: 1, "delete_indices": delete
2017-06-22 23:00:59,769 ERROR     Failed to complete action: delete_indices.  <class 'curator.exceptions.ActionError'>: Field "zephyr_timestamp" not found in index "logstash-dd.dcs_owner_log"

Please help me to be specific exact index name to perform delete option.

4

i have indexed value for 3 months like jan,feb,mar. I want to delete the entire feb month index value from index. How could i perform that using curator.

Index name like "logstash-dd.name_log"

5

Okay, I'm a bit confused. The configuration you shared has:

      source: field_stats
      field: 'log_timestamp'

But the error message indicates a field called zephyr_timestamp. The bottom line is that when using source: field_stats, whatever you select for field must be in all of the indices being acted on. If that field is not found in even one of the indices to be acted on, an exception like the one you shared will happen.

If the values for all three months are all in one index, then you cannot use Curator to delete them. Curator does not delete selective data from indices. It can only delete entire indices.

6

I got your point, having one more doubt please clarify it.

I want to delete old record for example need to keep last 90 days of records in my index. How could i perform that?

I tried using age filtertype but it delete entire index. help me on this

Index name will not contain timestamp value.

- filtertype: age
  source: field_stats
  field: '@timestamp'     
  stats_result: min_value 
  direction: older
  unit: days
  unit_count: 90
7

Curator only does index-level management. You can only use it to delete entire indices.

Elasticsearch offers a delete-by-query plugin. You should ask a separate question for that.

8

Thanks for your reply

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.