Index optimization fails to merge all segments

Elastic Stack Elasticsearch
1

Hello all,

We are running a production Elasticsearch 1.7.4 cluster with time-based data sitting in monthly indexes.

Indexes with data from previous months are very static and don't get written to, so we've decided to optimize them.

We ran the optimization on several indexs with default merge settings and the following optimize request parameters:

max_num_segments = 1
flush = true

Immediately after sending the API request, we get this response:

{"_shards":{"total":70,"successful":70,"failed":0}}

instead of getting blocked as expected.

After the request is sent, we can see merges start to occur, and the deleted documents count starts declining. Eventually the merges stop but the deleted documents count is still far greater than 0, and there are still hundreds if not thousands of segments in the index.

We've also noticed that sometimes another optimize request is necessary for the disk space to actually be cleared.

Is there something that we're missing? Why is optimize behaving unexpectedly?

Thanks for your help,
Alon

2

Are you writing to this index while the optimise is happening?

3

No, we don't
These are "old" indices (previous months)

4

We've resorted to reindexing since nothing seems to fix the issues I've raised in the original post. If anyone has any ideas or suggestions to resolve our issue, they would be greatly appreciated.