Error with curator delete_index

Elastic Stack Elasticsearch
1

Good morning,

I wanna delete the older index from kibana and I wanna user curator.

First , I've get the index that I've in kibana

curl -XGET 'localhost:9200/_cat/indices?v&pretty'
health status index           uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   twitter-traffic vFCR7_41QHyKgI0J8oPkbw   5   1       9924            0       51mb           51mb
yellow open   pan-traffic     1dIt-MZbTmeARb-a5YAsSQ   5   1  139262798            0    112.6gb        112.6gb
yellow open   .kibana         KOgWNAvPQ467H1d8s7-wlg   1   1         47            2    149.1kb        149.1kb

My action file is the following

---
actions:
  1:
    action: delete_indices
    description: >-
      Delete indices older than 45 days (based on index name), for pan-traffic
      prefixed indices. Ignore the error if the filter does not result in an
      actionable list of indices (ignore_empty_list) and exit cleanly.
    options:
      ignore_empty_list: True
      timeout_override:
      continue_if_exception: False
      disable_action: False
    filters:
    - filtertype: pattern
      kind: prefix
      value: pan-traffic,twitter-traffic
      exclude:
    - filtertype: age
      source: name
      direction: older
      timestring: '%Y.%m.%d'
      unit: days
      unit_count: 45

The config file

client:
  hosts:
    - 127.0.0.1
  port: 9200
  url_prefix:
  use_ssl: False
  certificate:
  client_cert:
  client_key:
  ssl_no_validate: False
  http_auth:
  timeout: 30
  master_only: False

the execution of the following command is the following

2017-08-16 10:10:37,488 INFO Preparing Action ID: 1, "delete_indices"
2017-08-16 10:10:37,503 INFO Trying Action ID: 1, "delete_indices": Delete indices older than 45 days (based on index name), for pan-traffic prefixed indices. Ignore the error if the filter does not result in an actionable list of indices (ignore_empty_list) and exit cleanly.
2017-08-16 10:10:37,569 INFO Skipping action "delete_indices" due to empty list: <class 'curator.exceptions.NoIndices'>
2017-08-16 10:10:37,569 INFO Action ID: 1, "delete_indices" completed.
2017-08-16 10:10:37,569 INFO Job completed.

any ideas whats wrong?

kindly regards

2

your age filter is wrongly configured

in your cofiguration it looks for a timstamp in the name of the index which you have none

you will have to use creation_date based age filter through setting source to "creation_date" and removing timestring

or you could use field_stats based filtering which would filter all indices in which the newest event is past the configured age

  • filtertype: age
    source: field_stats
    direction: older
    unit: days
    unit_count: 3
    field: '@timestamp'
    stats_result: max_value

For reference: https://www.elastic.co/guide/en/elasticsearch/client/curator/current/filtertype_age.html

principal question is why you would like to delete older indices if your indices are not timebased created?

1 Like
3

Hi, all the index are timestamp based:

actions:
  1:
    action: delete_indices
    description: >-
      Delete indices older than 45 days (based on index name), for pan-traffic
      prefixed indices. Ignore the error if the filter does not result in an
      actionable list of indices (ignore_empty_list) and exit cleanly.
    options:
      ignore_empty_list: True
      timeout_override:
      continue_if_exception: False
      disable_action: False
    filters:
    - filtertype: pattern
      kind: prefix
      value: pan-traffic
    - filtertype: age
      source: field_stats
      direction: older
      unit: days
      unit_count: 45
      field: '@timestamp'
      stats_result: max_value

Then if I remove the

- filtertype: pattern
          kind: prefix
          value: pan-traffic

It's works except with the following command Failed to complete action: delete_indices. <class 'curator.exceptions.ActionError'>: Field "@timestamp" not found in index ".kibana"

How can i exlude this index?

4

You could add a second filter pattern that excludes the .kibana index like

  • filtertype: pattern

    kind: prefix
value: .kibana
exclude: True

5

Many thanks

I'm doing in other way

  • filtertype: kibana
    exclude: True
6

yeh i hadn't thought of that filter it is realy the better way because it not only excludes .kibana but also some other meta indices

could you mark a solution?

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.