Cluster Rebalancing

Elastic Stack Elasticsearch
1

Hi All,

I'm using ILM policies and as part of the Warm-Phase indices should be shrinked. As you know, for that all of the shards of an index must be reallocated to a single node, which unfortunately quite often fails. It fails for the following error returned by: GET /_cluster/allocation/explain

_the node is above the low watermark cluster setting [cluster.routing.allocation.disk.watermark.low=85%], using more disk space than the maximum allowed [85.0%]_

The point is, that I have plenty of disk space available on other nodes, but due to disk allocation rules, Elasticsearch does not start to reallocate shards, just because the low disk watermark is reached.

This is annoying, as it stops the shrink process quite often and I have to manually reallocate shards away from the Low-Disk-Watermark node.

Do you have any recommendations how to make this process more stable? Perhaps telling Elasticsearch to start re-balancing already on Low-Disk-Watermark.

Thanks!
Chris

2

As a workaround I put the High-Disk watermark very close to the Low-disk watermark to force Elasticsearch to move shards away from the Low-Disk-Node. This helped, but I'm open for any other recommendation.

3

Can you increase the watermarks?

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.