How to explicitly trigger /disable merge process?


(venkata nagaraju buddarapu) #1

Hi,

how to explicitly trigger /disable merge process ?


(Joshua Rich) #2

You can't completely disable the merge process and you wouldn't want to anyway. Merging, although an expensive operation, helps keep your segment counts and sizes constrained which if not checked would cause you worse performance issues than the merge process itself.

There are a number of settings you can use to change the merge process, but we generally do not recommend changing these. You can explicitly force a merge with the optimize API endpoint (some of the merge settings do not apply to the optimize call).

What's the problem you are trying to solve here? There might be a better way to help than trying to "tune" merging.


(venkata nagaraju buddarapu) #3

I dont mean to say I want to disable completely disable merge .

As per my understanding and reading, I want to disable merge before large ingestion /updates starts and enable it back when large ingestion /updates are done .This way I can give my entire disk to ingestion /updates .From marvel what I found is ingestion /update speed slows down as soon as merge starts and also merge consume majority of the memory


(Joshua Rich) #4

Rather than changing the merging settings, I would suggest you look at any or all of the following:

  • Use the bulk api through whatever client you are using and tune it for optimum performance. This usually significantly improves indexing rates.
  • Increase index.refresh_interval at the cost of having more stale search results.

(Jason Wee) #5

well, i tried to call optimize api before, but the segment stay the same even after a while. any idea why?


(Joshua Rich) #6

Note the optimise call is blocking, it can take a long time to complete. My suggestion would be to try explicitly setting the max_num_segments parameter and performing an optimise on a single index to observe if it makes a difference:

curl -XPOST 'http://localhost:9200/someindex/_optimize?max_num_segments=2'

(Jason Wee) #7

i tried it, current segment was around a thousand and with max_num_segments to 900, after the curl is complete, it is still showing around a thousand.


(Joshua Rich) #8

Is that per index or over your entire cluster? It seems like an awful lot...


(Jason Wee) #9

yea i know... have to really find way to reduce the segment count. thanks anyway


(system) #10