# How to speed up the segment merging at night?

**URL:** https://discuss.elastic.co/t/how-to-speed-up-the-segment-merging-at-night/271267
**Category:** Elasticsearch
**Created:** [April 26, 2021, 2:27pm UTC](https://discuss.elastic.co/t/how-to-speed-up-the-segment-merging-at-night/271267 "2021-04-26T14:27:05Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![wangxr1985](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wangxr1985/32/117798_2.png) [@wangxr1985](https://discuss.elastic.co/u/wangxr1985)
#### Post date: [April 26, 2021, 2:27pm UTC](https://discuss.elastic.co/t/how-to-speed-up-the-segment-merging-at-night/271267/1 "2021-04-26T14:27:05Z")

</div>

The ES version is 7.11.1

I use ilm to manage my log indices  
hot phase: rollover  
warm phase (12 hours from rollover): forcemerge、read only、data allocation  
cold phase (5 days from rollover): freeze  
delete phase(10 days from rollover): delete

The average size of the shard is 40gb.  
In the warm phase I set the max\_num\_segments to 5 for forcemerge.

The issue is that there are too large size of data to do forcemerge, it processes slowly, all jobs can not be done in time, so all indices stuck at forcemerge step, can not be freezed.

The infomation of "\_cat/thread\_pool/force\_merge?v" is:

node\_name name active queue rejected  
012065 force\_merge 1 132 0  
008076 force\_merge 1 94 0  
010060 force\_merge 1 169 0  
012074 force\_merge 1 148 0  
012055 force\_merge 1 134 0

I wonder if there is a way to speed up the segment merging at night(0:00-7:00) everyday.  
I tried to change the thread\_pool.force\_merge.size but it is not dynamically updateable.

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [April 26, 2021, 10:20pm UTC](https://discuss.elastic.co/t/how-to-speed-up-the-segment-merging-at-night/271267/2 "2021-04-26T22:20:43Z")

</div>

The best way is to allocate more hardware resources for it, specifically faster disk.

There's no other magic switches here.

---

<div class="post-metadata">

### Author: ![wangxr1985](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wangxr1985/32/117798_2.png) [@wangxr1985](https://discuss.elastic.co/u/wangxr1985)
#### Post date: [April 27, 2021, 3:22am UTC](https://discuss.elastic.co/t/how-to-speed-up-the-segment-merging-at-night/271267/3 "2021-04-27T03:22:16Z")

</div>

Can I set the thread\_pool.force\_merge.size of each warm data node to a value larger than 1 to speed up the forcemerge?  
And can I do the I/O throttling in the daytime and disable it at night?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 25, 2021, 3:22am UTC](https://discuss.elastic.co/t/how-to-speed-up-the-segment-merging-at-night/271267/4 "2021-05-25T03:22:53Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
