How to explicitly trigger /disable merge process?

Elastic Stack Elasticsearch
1

Hi,

how to explicitly trigger /disable merge process ?

2

You can't completely disable the merge process and you wouldn't want to anyway. Merging, although an expensive operation, helps keep your segment counts and sizes constrained which if not checked would cause you worse performance issues than the merge process itself.

There are a number of settings you can use to change the merge process, but we generally do not recommend changing these. You can explicitly force a merge with the optimize API endpoint (some of the merge settings do not apply to the optimize call).

What's the problem you are trying to solve here? There might be a better way to help than trying to "tune" merging.

3

I dont mean to say I want to disable completely disable merge .

As per my understanding and reading, I want to disable merge before large ingestion /updates starts and enable it back when large ingestion /updates are done .This way I can give my entire disk to ingestion /updates .From marvel what I found is ingestion /update speed slows down as soon as merge starts and also merge consume majority of the memory

4

Rather than changing the merging settings, I would suggest you look at any or all of the following:

  • Use the bulk api through whatever client you are using and tune it for optimum performance. This usually significantly improves indexing rates.
  • Increase index.refresh_interval at the cost of having more stale search results.
5

well, i tried to call optimize api before, but the segment stay the same even after a while. any idea why?

6

Note the optimise call is blocking, it can take a long time to complete. My suggestion would be to try explicitly setting the max_num_segments parameter and performing an optimise on a single index to observe if it makes a difference:

curl -XPOST 'http://localhost:9200/someindex/_optimize?max_num_segments=2'
7

i tried it, current segment was around a thousand and with max_num_segments to 900, after the curl is complete, it is still showing around a thousand.

8

Is that per index or over your entire cluster? It seems like an awful lot...

9

yea i know... have to really find way to reduce the segment count. thanks anyway