How to speed up the segment merging at night?

Elastic Stack Elasticsearch
1

The ES version is 7.11.1

I use ilm to manage my log indices
hot phase: rollover
warm phase (12 hours from rollover): forcemerge、read only、data allocation
cold phase (5 days from rollover): freeze
delete phase(10 days from rollover): delete

The average size of the shard is 40gb.
In the warm phase I set the max_num_segments to 5 for forcemerge.

The issue is that there are too large size of data to do forcemerge, it processes slowly, all jobs can not be done in time, so all indices stuck at forcemerge step, can not be freezed.

The infomation of "_cat/thread_pool/force_merge?v" is:

node_name name active queue rejected
012065 force_merge 1 132 0
008076 force_merge 1 94 0
010060 force_merge 1 169 0
012074 force_merge 1 148 0
012055 force_merge 1 134 0

I wonder if there is a way to speed up the segment merging at night(0:00-7:00) everyday.
I tried to change the thread_pool.force_merge.size but it is not dynamically updateable.

2

The best way is to allocate more hardware resources for it, specifically faster disk.

There's no other magic switches here.

3

Can I set the thread_pool.force_merge.size of each warm data node to a value larger than 1 to speed up the forcemerge?
And can I do the I/O throttling in the daytime and disable it at night?

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.