Automatically removing index

[gavenkoa (Oleksandr Gavenko)]

Docs says you need to remove unused (yeah, old) indexes: https://www.elastic.co/guide/en/elasticsearch/guide/current/retiring-data.html

There is https://www.elastic.co/guide/en/elasticsearch/client/curator/current/installation.html to help with.

Is there an option for index to be removed automatically by ES after some time passed?

Currently we don't have time to learn tools better and stripping month history in order to prevent storage outage is good enough.

[dadoonet (David Pilato)]

Curator is the way to go.

Otherwise run your own DELETE index-name command every day. It’s a simple call you can do from your application

[gavenkoa (Oleksandr Gavenko)]

I end with curator:

bash# wget https://packages.elastic.co/curator/5/centos/6/Packages/elasticsearch-curator-5.3.0-1.x86_64.rpm
bash# rpm -i elasticsearch-curator-5.3.0-1.x86_64.rpm
bash# curator_cli show_indices

192.168.10.33:/etc/cron.daily/curator-purge-old-indexes

#!/bin/sh
/usr/bin/curator curator --config /etc/elasticsearch/curator-cfg.yml /etc/elasticsearch/curator-action-delete.yml

/etc/elasticsearch/curator-action-delete.yml

actions:
  1:
    action: delete_indices
    description: >-
      Delete old indexes.
    options:
      ignore_empty_list: True
      timeout_override:
      continue_if_exception: False
      disable_action: False
    filters:
    - filtertype: pattern
      kind: prefix
      value: logstash-
      exclude:
    - filtertype: age
      source: name
      direction: older
      timestring: '%Y.%m.%d'
      unit: days
      unit_count: 30
  2:
    action: delete_indices
    description: >-
      Delete old indexes.
    options:
      ignore_empty_list: True
      timeout_override:
      continue_if_exception: False
      disable_action: False
    filters:
    - filtertype: pattern
      kind: prefix
      value: sa-
      exclude:
    - filtertype: age
      source: name
      direction: older
      timestring: '%Y-%m-%d'
      unit: days
      unit_count: 30

/etc/elasticsearch/curator-cfg.yml

client:
  hosts:
    - 192.168.10.32
    - 192.168.10.33
  port: 9200

To test:

/usr/bin/curator curator --dry-run --config /etc/elasticsearch/curator-cfg.yml /etc/elasticsearch/curator-action-delete.yml