We are using Elastic for a year now, and we're very pleased about it.
We have 2 nodes sitting on different servers with estimated size of 250GB of data sitting one index with 6 shards each.
every now and then we will run out of disk space and I would delete some doc's and run the optimize with only_expunge_deletes=true command.
but now we reached a point were that command needs addition disk space to make the lucene index marge.
So it failed on one of our nodes , but the good news is that we do succeeded in the other node and right now our stat is Yellow.
My question is :
Because the failed node has no disk space can I manually delete shards folders, hopping that the working node will reallocate the missing shard (which was already space reduced) back ?
If not which files can I delete in order to give the miserable node enough space to recover and complete the optimize command ?
Thanks in advance.