Elastic search Cluster health due to disk space

[mark.penner (mark)]

Hi,

I am currently experiencing an issue on my elastic cluster due to low disk space.

"explanation" : "the node is above the low watermark cluster setting [cluster.routing.allocation.disk.watermark.low=85%], using more disk space than the maximum allowed [85.0%], actual free: [9.737415872042689%]"

I have located the files in question and it looks like the following is taking up the space.

/home/????/elasticsearch-6.5.1/data/nodes/0/indices

Forgive my naivety but could any one recommend a safe way of clearing out these indices or freeing up some space?

Many thanks,

[Bernt_Rostad]

@mark.penner

actual free: [9.737415872042689%]

It looks like you've got less than 10% free disk space, which means you cannot do a force merge to remove deleted documents (because the merge operation will write new segments before removing the old).

@mark.penner

Forgive my naivety but could any one recommend a safe way of clearing out these indices or freeing up some space?

With insufficient space for a force merge you have three options left, that I can think of:

1. Increase the disk space on the almost full server.
2. Add more data nodes to your cluster to spread out the index shards on more nodes.
3. Delete one large index (or more) to free up disk space immediately.

Good luck!