Curator, delete old indices when cluster disk threshold is hit

Elastic Stack Elasticsearch
1

Reading the curator documentation I can create a filter that is hit if total size of the cluster goes over X amount, and I can delete old indices if they are over a certain size.

I can not for the life of me figure out if I can automatically delete indices with curator if free disk space is below X amount. What I will be left to do is create my own little program that does it....

2

In most use cases this does not make sense, for a few reasons:

  1. Shard allocation might mean that some nodes are more used than others, resulting in full disk notifications
  2. Shard count per node can drastically affect cluster and JVM health. Deleting when only x gigabytes of free space remains potentially results in these issues.

It is for these reasons specifically that Curator does not support this methodology. It has been oft requested, but after the initial discussion with the Elasticsearch team, it is not recommended. Even the regular delete by space is not particularly ideal, though filter chaining makes it more so.

3

Please also see this answer to a very similar question.

4

Thanks that explains the big picture of why.

My use case is much smaller where there is only one node with Elasticsearch so I just want curator to delete old indices when space is low until space thresholds are ok. Something that is easily scriptable though since Curator does not support it.

5

I quote from my post in the above-linked page (italics added):

For these reasons, deleting indices exceeding a disk space watermark is not likely to be added to Curator until it becomes a practice marked as either acceptable or recommended by the core Elasticsearch developers. While you might be able to successfully argue that your particular use case may be a safe one in which to use this approach, providing it as an out-of-the-box feature in Curator makes it look like it is not only acceptable, but completely normal. I just can't do that.

I'm sorry that Curator isn't meeting your needs here, but this is a use case I simply cannot bundle into Curator until it's an approved approach for all clusters.

6

I understand. To make a feature for this work curator would have to know how much free space is in each node and how large each shard is. Then make some calculations on how much space is actually left and how it might affect cluster health. Not easy for big clusters.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.