Is it best practice to call _forcemerge on active index?

(Xudong You) #1

I am going to optimize my indices daily with automatic tool which will simply call _forcemerge?max_num_segments=1 all indices one by one. During the optimizing, there are still querying and indexing on all indices.

Is it best practice to merge the active index with _forcemerge?max_num_segments=1?

(Mark Walkom) #2

It is not, because segment counts are changing.

(Xudong You) #3

So any approaches to optimize the active index? Our search service must be 24x24 alive and the index is queried and indexed all the time, so cannot close, optimize then open.

(Nik Everett) #4

_forcemerge is nice for indexes that you know you are done writing to. Otherwise you just continue to make your changes to the index, relying on the background merge process to clean up the deleted documents as it goes.

(Xudong You) #5

If we can temporarily stop the indexing, is it OK to do _forcemerge on index which are still being heavily queried?
Per our performance testing, seems _forcemerge has no obvious impact on query. But want to double confirm.

(David Pilato) #6

There is no need to call forcemerge if you are going to append new documents after that.
What problem are you trying to solve here?

(Xudong You) #7

In our previous testing, we found low query perf on a certain index, and after we called _forcemerge?max_num_segments=1, this issue was gone.

So I think maybe we can optimize the index regularly to avoid the potential bad query perf in future.

(Nik Everett) #8

Calling forcemerge on an index you are actively updating will make it
perform better for a short time and then much worse. It is only a good
choice if you aren't going to modify the index again.

(Xudong You) #9

Per your answer, optimize on active index will make it "perform better for a short time and then much worse".
Why "then much worse" ?

Also, if I can make sure stop the updating before forcemerge and then resume the updating after forcemerge done, should be no perf impact, right?

(system) #10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.