# Force\_merge Thread\_pool

**URL:** https://discuss.elastic.co/t/force-merge-thread-pool/245108
**Category:** Elasticsearch
**Created:** [August 16, 2020, 12:30am UTC](https://discuss.elastic.co/t/force-merge-thread-pool/245108 "2020-08-16T00:30:50Z")
**Posts on this page:** 10
**Page:** 1

<div class="post-metadata">

### Author: ![Thek10patil](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thek10patil/32/73940_2.png) [@Thek10patil](https://discuss.elastic.co/u/Thek10patil)
#### Post date: [August 16, 2020, 12:30am UTC](https://discuss.elastic.co/t/force-merge-thread-pool/245108/1 "2020-08-16T00:30:50Z")

</div>

Hi Everyone,

I need to run `force_merge` frequently to have `max_num_segments = 1`. It takes lot of time. I want to make force\_merge faster. What are the possible option do I have?

I read and found that max size of threads available for force\_merge is 1. I changed that to 4 as I have 8 thread machine by changing `elasticsearch.yml` config file. I can see the respective change as well in node setting.

I checked during force\_merge execution, it shows as 1 active thread even though current thread size is 4 for force\_merge.

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [August 16, 2020, 5:52am UTC](https://discuss.elastic.co/t/force-merge-thread-pool/245108/2 "2020-08-16T05:52:56Z")

</div>

How many indices and shards are you forcemerging? Why do you need to run it so often? What is the size of the cluster?

---

<div class="post-metadata">

### Author: ![Thek10patil](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thek10patil/32/73940_2.png) [@Thek10patil](https://discuss.elastic.co/u/Thek10patil)
#### Post date: [August 17, 2020, 5:38pm UTC](https://discuss.elastic.co/t/force-merge-thread-pool/245108/3 "2020-08-17T17:38:16Z")

</div>

I am planning to use Ultrawarm AWS feature. For that I wanted to run force merge first on all the available indices.

Currently force-merge uses one thread only. I have added more threads using yml file but while running force merge it is showing only one active thread.

---

<div class="post-metadata">

### Author: ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)
#### Post date: [August 17, 2020, 7:20pm UTC](https://discuss.elastic.co/t/force-merge-thread-pool/245108/4 "2020-08-17T19:20:20Z")

</div>

> [@Thek10patil](#):
>
> I am planning to use Ultrawarm AWS feature

You will likely need to approach AWS support for help with this question, AWS Elasticsearch is different from the official Elasticsearch that we support here.

---

<div class="post-metadata">

### Author: ![Thek10patil](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thek10patil/32/73940_2.png) [@Thek10patil](https://discuss.elastic.co/u/Thek10patil)
#### Post date: [August 17, 2020, 8:12pm UTC](https://discuss.elastic.co/t/force-merge-thread-pool/245108/5 "2020-08-17T20:12:25Z")

</div>

Hey David,

I know that feature is from AWS. But I am looking to make Force\_Merge faster.

For now force\_merge to `max_segments = 1` with default setting taking 25 mins for 50GB shard which has around 45 segments. I want to make it faster.

What will be a good way to do this?

Thanks in advance.

---

<div class="post-metadata">

### Author: ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)
#### Post date: [August 17, 2020, 8:20pm UTC](https://discuss.elastic.co/t/force-merge-thread-pool/245108/6 "2020-08-17T20:20:16Z")

</div>

Can you reproduce the issue outside of AWS Elasticsearch? The only people who have access to the AWS Elasticsearch code are AWS themselves, and it's pretty tricky to debug this kind of thing without access to the code that you're actually running.

---

<div class="post-metadata">

### Author: ![Thek10patil](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thek10patil/32/73940_2.png) [@Thek10patil](https://discuss.elastic.co/u/Thek10patil)
#### Post date: [August 17, 2020, 9:11pm UTC](https://discuss.elastic.co/t/force-merge-thread-pool/245108/7 "2020-08-17T21:11:58Z")

</div>

I am not sure if I can reproduce it but for now I am looking for parameters which will help to make `force_merge` faster.

Parameters such as

- threads assigned for force\_merge though `THREAD_POOL` node setting
- index.merge.scheduler.max\_thread\_count

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [August 18, 2020, 5:23am UTC](https://discuss.elastic.co/t/force-merge-thread-pool/245108/8 "2020-08-18T05:23:16Z")

</div>

If I recall correctly each shard is forcemerged in a single thread. Increasing the thread pool allows multiple shards to be forcemerged in parallel but would as far as I know not speed up a single forcemerge. I am not aware of any setting that would speed it up, but the time it takes is however generally proportional to the shard size so you could try to reduce the shard size.

---

<div class="post-metadata">

### Author: ![Thek10patil](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thek10patil/32/73940_2.png) [@Thek10patil](https://discuss.elastic.co/u/Thek10patil)
#### Post date: [September 3, 2020, 10:40pm UTC](https://discuss.elastic.co/t/force-merge-thread-pool/245108/9 "2020-09-03T22:40:46Z")

</div>

Yes you are correct. Increasing thread pools allows to run FM on multiple shards.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 1, 2020, 10:40pm UTC](https://discuss.elastic.co/t/force-merge-thread-pool/245108/10 "2020-10-01T22:40:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
