Remove old index automatically

Elastic Stack Elasticsearch
,
1

Hello All,

I need to remove all the indexes older than 6 days automatically, I tried to create a lifecycle in Kibana but it didn't work fine, so i saw an option called Curator but for some reason, the curator can't find the indexes.

The indexed are:

curator_cli --config /home/elkadmin/curator/curator-cfg.yml show_indices | grep cisco-asa
      cisco-asa-2020.06.23
      cisco-asa-2020.06.24
      cisco-asa-2020.06.25
      cisco-asa-2020.06.26
      cisco-asa-2020.06.27
      cisco-asa-2020.06.28
      cisco-asa-2020.06.29
      cisco-asa-2020.06.30

I installed version 5.8 in one of my data nodes.
This is the config file:

curator-cfg.yml
    client:
      hosts:
        - 10.70.4.142
        - 10.70.4.143
      port: 9200
    logging:
      loglevel: INFO
      logfile:
      logformat: default
      blacklist: ['elasticsearch', 'urllib3']

Action - curator-action-delete.yml

    actions:
      1:
        action: delete_indices
        description: >-
          Delete old indexes.
        options:
          ignore_empty_list: True
          timeout_override: 30
        filters:
        - filtertype: pattern
          kind: prefix
          value: cisco-asa-
        - filtertype: age
          source: name
          direction: older
          timestring: '%Y.%m.%d'
          unit: days
          unit_count: 5

output

    /usr/bin/curator /home/elkadmin/curator/curator-action-delete.yml --config /home/elkadmin/curator/curator-cfg.yml
    2020-06-30 09:51:27,976 INFO      Preparing Action ID: 1, "delete_indices"
    2020-06-30 09:51:27,976 INFO      Creating client object and testing connection
    2020-06-30 09:51:27,980 INFO      Instantiating client object
    2020-06-30 09:51:27,981 INFO      Testing client connectivity
    2020-06-30 09:51:27,984 INFO      Successfully created Elasticsearch client object with provided settings
    2020-06-30 09:51:27,987 INFO      Trying Action ID: 1, "delete_indices": Delete old indexes.
    2020-06-30 09:51:28,222 INFO      Skipping action "delete_indices" due to empty list: <class 'curator.exceptions.NoIndices'>
    2020-06-30 09:51:28,222 INFO      Action ID: 1, "delete_indices" completed.
    2020-06-30 09:51:28,222 INFO      Job completed.
2

This is almost certainly because the indices are associated with an ILM policy. By default, Curator will not interact with an index associated with an ILM policy, so you have to manually tell Curator to ignore that with allow_ilm_indices: true in your option block:

        options:
          allow_ilm_indices: true
          ignore_empty_list: True
          timeout_override: 30
3

It worked !! thank you so much

/usr/bin/curator /home/elkadmin/curator/curator-action-delete.yml --config /home/elkadmin/curator/curator-cfg.yml
2020-06-30 10:22:48,904 INFO      Preparing Action ID: 1, "delete_indices"
2020-06-30 10:22:48,904 INFO      Creating client object and testing connection
2020-06-30 10:22:48,908 INFO      Instantiating client object
2020-06-30 10:22:48,908 INFO      Testing client connectivity
2020-06-30 10:22:48,912 INFO      Successfully created Elasticsearch client object with provided settings
2020-06-30 10:22:48,915 WARNING   allow_ilm_indices: true
2020-06-30 10:22:48,915 WARNING   Permitting operation on indices with an ILM policy
2020-06-30 10:22:48,915 INFO      Trying Action ID: 1, "delete_indices": Delete old indexes.
2020-06-30 10:22:49,137 INFO      Deleting 3 selected indices: ['cisco-asa-2020.06.24', 'cisco-asa-2020.06.23', 'cisco-asa-2020.06.25']
2020-06-30 10:22:49,137 INFO      ---deleting index cisco-asa-2020.06.24
2020-06-30 10:22:49,137 INFO      ---deleting index cisco-asa-2020.06.23
2020-06-30 10:22:49,137 INFO      ---deleting index cisco-asa-2020.06.25
2020-06-30 10:22:51,737 INFO      Action ID: 1, "delete_indices" completed.
2020-06-30 10:22:51,737 INFO      Job completed.
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.