Curator Questions: Scheduling and Round-Robin

Elastic Stack Elasticsearch
1

So I have two questions that I have not been able to find an answer for yet. First and most annoying, is I am unable to schedule Curator to on a daily basis. I have a cron job setup as follows:

20 0 * * * /bin/curator --host es-1 close indices --older-than 30 --time-unit days --timestring '%Y.%m.%d'
30 0 * * * /bin/curator --host es-1 delete indices --older-than 60 --time-unit days --timestring '%Y.%m.%d'

The same command runs fine when I run it manually from the command line:

sudo curator --host es-1 close indices --older-than 30 --time-unit days --timestring '%Y.%m.%d'
sudo curator --host es-1 delete indices --older-than 60 --time-unit days --timestring '%Y.%m.%d'

Any idea what im missing??

Second question, as you can see from the above commands im pointing directly to the host es-1. Is there anyway of modifying this so it will pick any of the hosts in the cluster? I want the command to run even if one of the hosts is down.

2

Frequently, cron can have issues with percent signs % in commands. Sometimes it doesn't pick up all of the flags because it doesn't like long command lines. I recommend a wrapper script instead.

Curator shouldn't need to be run as root, by the way. It only needs access to a client host/IP address.

As far as host round robin goes, this feature is in the development version of Curator 4: https://www.elastic.co/guide/en/elasticsearch/client/curator/4.0/configfile.html#hosts

It will not be back-ported to Curator 3, as the structure of Curator's configuration is so different.

3

Cool so ill move the commands into a bash script and instead call the bash script from my cron job.

Good news about the round robin, ill keep an eye on it in a future release.

4

This should work, right?

20 0 * * * /usr/local/bin/elasticsearch-curator.sh

With the script containing the following:

#closes open indices after 30 days
curator --host es-1 close indices --older-than 30 --time-unit days --timestring '%Y.%m.%d'

#delete closed indices after 60 days
curator --host es-1 delete indices --older-than 60 --time-unit days --timestring '%Y.%m.%d'

Is there a way to manually invoke a cron job?

5

Should work, yes. But all of the STDOUT will be emailed to you. Perhaps you will want to set a --logfile in these lines. You could temporarily add a --dry-run flag and set an additional cron entry a few minutes into the future and see what the log files show.

6

Just to close out this request, Aarons suggestion to run the commands out of a bash script worked perfectly.