Es 5.6.x force merege question about max_num_segments/max_merge_count/max_thread_count

zqc0512 · July 3, 2018, 1:49am

i see the doc have setting with the indices about force merege
index.merge.scheduler.max_merge_count
index.merge.scheduler.max_thread_count
with not ssd the docs give index.merge.scheduler.max_thread_count to "1" ,and the
index.merge.scheduler.max_merge_count aslo need set to 1

the question is :

1.the max_num_segments in the flow cmd when force merege is same as max_merge_count?
curl -XPOST "http://xxxxx:9200/xxxx/_forcemerge?max_num_segments=1
2.have many segments better with index search or small segment better with index search?

zqc0512 · July 3, 2018, 2:00am

with monitoring
the one index have so many segment count

zqc0512 · July 3, 2018, 7:22am

some one can help me?

zqc0512 · July 3, 2018, 3:13pm

i test it. can't find how setting about it........

zqc0512 · July 5, 2018, 2:17am

there no doc about this..
i see the soure code about this .

github.com

elastic/elasticsearch/blob/5.6/core/src/main/java/org/elasticsearch/index/MergeSchedulerConfig.java#L60


*     move higher to allow merges to keep up with ongoing indexing.
* </ul>
*/
public final class MergeSchedulerConfig {


   public static final Setting<Integer> MAX_THREAD_COUNT_SETTING =
       new Setting<>("index.merge.scheduler.max_thread_count",
           (s) -> Integer.toString(Math.max(1, Math.min(4, EsExecutors.boundedNumberOfProcessors(s) / 2))),
           (s) -> Setting.parseInt(s, 1, "index.merge.scheduler.max_thread_count"), Property.Dynamic,
           Property.IndexScope);
   public static final Setting<Integer> MAX_MERGE_COUNT_SETTING =
       new Setting<>("index.merge.scheduler.max_merge_count",
           (s) -> Integer.toString(MAX_THREAD_COUNT_SETTING.get(s) + 5),
           (s) -> Setting.parseInt(s, 1, "index.merge.scheduler.max_merge_count"), Property.Dynamic, Property.IndexScope);
   public static final Setting<Boolean> AUTO_THROTTLE_SETTING =
       Setting.boolSetting("index.merge.scheduler.auto_throttle", true, Property.Dynamic, Property.IndexScope);


   private volatile boolean autoThrottle;
   private volatile int maxThreadCount;
   private volatile int maxMergeCount;

only with +5 ...
i don't know which is better about index.merge.scheduler.max_merge_count and index.merge.scheduler.max_thread_count and max_num_segments for kibana search.
the index have 4.3b docs..

Bernt_Rostad · July 5, 2018, 7:38am

I'm not sure what your question is, but you'll find the documentation about _forcemerge and its request parameters here.

In general Elasticsearch will handle its own segment merging well, but it's sometimes useful to trigger a forcemerge as it will expunge deleted documents when merging two segments into one and therefore save you disk space.

But note that _forcemerge should not be called on an active index as that may result in huge (>5G) segments that will usually never be merged again and therefore could build up many deleted documents if that shard is still being updated.

zqc0512 · July 5, 2018, 8:15am

about the this i want to know about which number is better as flow
"index.merge.scheduler.max_merge_count": "20",
"index.merge.scheduler.max_thread_count": "2"
i want to know ,use kibana search the index before _forcemerge?max_num_segments=1 is good or after forcemerge?

and the max_num_segments can set as "2" or other greater than 1 number ,which is better for search index data?

system · August 2, 2018, 8:15am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.