Error message: the node is expected to continue to exceed the high disk watermark when these relocations are complete

Elastic Stack Elasticsearch
1

Hi!

There’s this scenario where my cluster elasticsearch in rke logs is showing the following message:
the node is expected to continue to exceed the high disk watermark when these relocations are complete. It keeps logging about it. What actions should I take ? And elasticsearch service is unavailable.

2

Hi @Mhvrke - it sounds like you have filled the disks quite a bit and hit the high disk watermark for a given node. The warning is telling it's trying to relocate shards, but after doing so, the threshold will still be surpassed.

How does your storage look for all your nodes?

You will want to either add more disk space, add more nodes, or remove old data that is no longer needed.

3

Are you still running with a shared disk volume as described in this thread? If that is the case I suspect all nodes will experience watermark issues at the same time and that relocating shards will just increase disk usage.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.