Creating Cron job for Close and Delete indices from elasticsearch


(Waruna) #1

I would like to share my Cron job for Close and Delete indices from elasticsearch 5.4

First installed elasticsearch-curator

then created /etc/elasticsearch-curator folder for config files
Created below files on that folder

  1. curator.yml (config file)
  2. action_close.yml (close action file)
  3. action_delete.yml (delete action file)
  4. curator_cron.sh (shell script for cron job)

curator.yml

client:
hosts:
- 127.0.0.1
port: 9200
url_prefix:
use_ssl: False
certificate:
client_cert:
client_key:
ssl_no_validate: False
http_auth:
timeout: 30
master_only: False

logging:
loglevel: INFO
logfile:
logformat: default
blacklist: ['elasticsearch', 'urllib3']

action_close.yml

actions:
1:
action: close
description: >-
Close indices older than 20 days (based on index name), for logstash-
prefixed indices.
options:
delete_aliases: False
disable_action: False
filters:
- filtertype: pattern
kind: prefix
value: logstash-
- filtertype: age
source: name
direction: older
timestring: '%Y.%m.%d'
unit: days
unit_count: 20

action_delete.yml

actions:
1:
action: delete_indices
description: >-
Delete indices older than 50 days (based on index name), for logstash-
prefixed indices. Ignore the error if the filter does not result in an
actionable list of indices (ignore_empty_list) and exit cleanly.
options:
ignore_empty_list: True
disable_action: False
filters:
- filtertype: pattern
kind: prefix
value: logstash-
- filtertype: age
source: name
direction: older
timestring: '%Y.%m.%d'
unit: days
unit_count: 50

curator_cron.sh

#!/bin/bash
#elasticsearch curator run daly
#Cron job
#30 2 * * * /etc/elasticsearch-curator/curator_cron.sh >>/var/log/curator.log 2>&1
#Close indices older than 20 days (based on index name), for logstash-
HOST=$(hostname)
start_time=$(date)
echo "$hostname Starting curator indices Close job: $start_time"
/usr/bin/curator --config /etc/elasticsearch-curator/curator.yml /etc/elasticsearch-curator/action_close.yml >>/var/log/curator.log 2>&1
stop_time=$(date)
echo "$hostname Stopping curator indices Close job: $stop_time"

#Delete indices older than 50 days (based on index name), for logstash- prefixed indices
start_time=$(date)
echo "$hostname Starting curator indices Delete job: $start_time"
/usr/bin/curator --config /etc/elasticsearch-curator/curator.yml /etc/elasticsearch-curator/action_delete.yml >>/var/log/curator.log 2>&1
stop_time=$(date)
echo "$hostname Stopping curator indices Delete job: $stop_time"

added ## curator_cron.sh for crontab runing every dat 2:30 am
crontab -e
30 2 * * * /etc/elasticsearch-curator/curator_cron.sh >>/var/log/curator.log 2>&1

note :-
no need curator_cron.sh file we can add
curator derectly crontab like below if you want


Curator example not taking action & w/error
(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.