Advice on force merge


(John T) #1

We have been seeing degraded search performance on our 2.3.2 cluster. In looking at our main index, some of the shards are more than 50% deleted documents, and there are 1,610 segments. This index has 360 million documents in 15 shards, and a large portion are updated often. I don't think that merging is able to keep up with the incoming/update rate. I need to address this, but I would like to merge out the old files asap. I had planned on doing this over the weekend, and i was going to pause writing to this index, and run a force merge, but...

  • I have read that with large shards / large # of segments can take a long time to run

  • There appears to be no way to monitor the status of the merge

  • There appears to be no way to cancel the force merge

In that I will have writes disabled, and search will probably be unavailable during the merge, I am hesitant to do this based on the bullets above (as I would want things back up and running in time to ensure all is well before start of day Monday). Do I have my information correct. Additional questions:

  1. I keep reading that you should only force merge on indexes you "no longer write to." Does that mean during the merge or at all? After the merge, I would need to open it back up for writing (understanding that I will likely get back to the same state eventually).

  2. Could I effectively do the same thing by reindexing to a new index? Though, I may have the same issues with monitoring /cancelling, right?

  3. Anything else I might consider that might speed regular live merging without degrading search performance further?

Thanks!


(Mark Walkom) #2

Ideally at all. If you want to merge and write, hold off writes until afterwards.

Yes.

Upgrade to 5.X, merges are much more intelligent.


(John T) #3

Am I correct in my (bullets) assumptions that there is no way to monitor and/or cancel a force merge?

thanks,
~john


(Aaron Mildenstein) #4

That is correct. Once started, there is no way to cancel a forceMerge. And there really isn't a way to check on its progress, either.


(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.