# Elastic Search max\_num\_segments in forcemerge

**URL:** https://discuss.elastic.co/t/elastic-search-max-num-segments-in-forcemerge/127472
**Category:** Elasticsearch
**Created:** [April 10, 2018, 11:44am UTC](https://discuss.elastic.co/t/elastic-search-max-num-segments-in-forcemerge/127472 "2018-04-10T11:44:37Z")
**Posts on this page:** 16
**Page:** 1

<div class="post-metadata">

### Author: ![raj.shah1593](https://avatars.discourse-cdn.com/v4/letter/r/8edcca/32.png) [@raj.shah1593](https://discuss.elastic.co/u/raj.shah1593)
#### Post date: [April 10, 2018, 11:44am UTC](https://discuss.elastic.co/t/elastic-search-max-num-segments-in-forcemerge/127472/1 "2018-04-10T11:44:37Z")

</div>

Hi,

I am currently using ES v6.1.2. I have an index of size 1.3 tb consisting of 16 shards (8 primaries).  
The primary shard size amounts to around 700GB.

I am planning to perform a forcemerge operation on the index.  
Any suggestions, what should be the max\_num\_segment value. As far as I know, by default, it takes the value as 1.

I want my search speed to improve. So what should be the ideal value for max\_num\_segments.

Thanks in advance 🙂

---

<div class="post-metadata">

### Author: ![raj.shah1593](https://avatars.discourse-cdn.com/v4/letter/r/8edcca/32.png) [@raj.shah1593](https://discuss.elastic.co/u/raj.shah1593)
#### Post date: [April 12, 2018, 11:18am UTC](https://discuss.elastic.co/t/elastic-search-max-num-segments-in-forcemerge/127472/2 "2018-04-12T11:18:58Z")

</div>

Hi,

Any update?  
Kindly help. Need to cater to this on an urgent basis.

Regards,  
Raj Shah

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [April 12, 2018, 12:27pm UTC](https://discuss.elastic.co/t/elastic-search-max-num-segments-in-forcemerge/127472/3 "2018-04-12T12:27:09Z")

</div>

1 is ok.  
But more than that you can probably do everything with fewer shards. Try with 1 shard only.  
The shrink API might help.

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [April 12, 2018, 12:35pm UTC](https://discuss.elastic.co/t/elastic-search-max-num-segments-in-forcemerge/127472/4 "2018-04-12T12:35:37Z")

</div>

How can the size of the primary shards be 700MB if the total size of the index across 16 shards is 1.3TB?

---

<div class="post-metadata">

### Author: ![raj.shah1593](https://avatars.discourse-cdn.com/v4/letter/r/8edcca/32.png) [@raj.shah1593](https://discuss.elastic.co/u/raj.shah1593)
#### Post date: [April 12, 2018, 12:37pm UTC](https://discuss.elastic.co/t/elastic-search-max-num-segments-in-forcemerge/127472/5 "2018-04-12T12:37:53Z")

</div>

Hey @Christian_Dahlqvist,

I am sorry. That should read 700GB.

Regards,  
Raj Shah

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [April 12, 2018, 12:50pm UTC](https://discuss.elastic.co/t/elastic-search-max-num-segments-in-forcemerge/127472/6 "2018-04-12T12:50:15Z")

</div>

That invalidates my answer.

---

<div class="post-metadata">

### Author: ![raj.shah1593](https://avatars.discourse-cdn.com/v4/letter/r/8edcca/32.png) [@raj.shah1593](https://discuss.elastic.co/u/raj.shah1593)
#### Post date: [April 12, 2018, 1:32pm UTC](https://discuss.elastic.co/t/elastic-search-max-num-segments-in-forcemerge/127472/7 "2018-04-12T13:32:03Z")

</div>

Hey @dadoonet,

Sorry for the typing error before.  
Just to confirm, should I go ahead with max\_seg\_count as 1?  
Or we should start with some higher number for this volume of data?

Regards,  
Raj Shah

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [April 12, 2018, 2:16pm UTC](https://discuss.elastic.co/t/elastic-search-max-num-segments-in-forcemerge/127472/8 "2018-04-12T14:16:55Z")

</div>

As long as you are not writing anymore to this index, I think that could be ok to set it to 1.  
@Christian_Dahlqvist WDYT?

---

<div class="post-metadata">

### Author: ![raj.shah1593](https://avatars.discourse-cdn.com/v4/letter/r/8edcca/32.png) [@raj.shah1593](https://discuss.elastic.co/u/raj.shah1593)
#### Post date: [April 12, 2018, 2:20pm UTC](https://discuss.elastic.co/t/elastic-search-max-num-segments-in-forcemerge/127472/9 "2018-04-12T14:20:02Z")

</div>

Hi @dadoonet,

We do perform write operation once everyday.

Regards,  
Raj Shah

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [April 12, 2018, 2:26pm UTC](https://discuss.elastic.co/t/elastic-search-max-num-segments-in-forcemerge/127472/10 "2018-04-12T14:26:57Z")

</div>

Do you batch update the index once a day or is it a continuous update?

---

<div class="post-metadata">

### Author: ![raj.shah1593](https://avatars.discourse-cdn.com/v4/letter/r/8edcca/32.png) [@raj.shah1593](https://discuss.elastic.co/u/raj.shah1593)
#### Post date: [April 12, 2018, 2:29pm UTC](https://discuss.elastic.co/t/elastic-search-max-num-segments-in-forcemerge/127472/11 "2018-04-12T14:29:21Z")

</div>

Hi @Christian_Dahlqvist,

It happens only once a day.

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [April 12, 2018, 2:37pm UTC](https://discuss.elastic.co/t/elastic-search-max-num-segments-in-forcemerge/127472/12 "2018-04-12T14:37:19Z")

</div>

Performing a forcemerge on shards that large will use up a lot of resources. I would recommend trying it once and see if it brings the benefits you are hoping for. I think it should be OK to set `max)num_segments` to 1, so that is what I would try with.

---

<div class="post-metadata">

### Author: ![raj.shah1593](https://avatars.discourse-cdn.com/v4/letter/r/8edcca/32.png) [@raj.shah1593](https://discuss.elastic.co/u/raj.shah1593)
#### Post date: [April 12, 2018, 2:40pm UTC](https://discuss.elastic.co/t/elastic-search-max-num-segments-in-forcemerge/127472/13 "2018-04-12T14:40:35Z")

</div>

Hi @Christian_Dahlqvist & @dadoonet ,

Thanks for your prompt response.  
Apart from the process being resource intensive, I would like to know if it has any other disadvantages.  
Since the data set is quite large, just want to be sure about it.

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [April 12, 2018, 2:43pm UTC](https://discuss.elastic.co/t/elastic-search-max-num-segments-in-forcemerge/127472/14 "2018-04-12T14:43:01Z")

</div>

It can end up taking a long time and use a lot of disk I/O, so could affect users while it is progressing. For that reason I would recommend trying it out in a test environment rather that trying it in production.

---

<div class="post-metadata">

### Author: ![raj.shah1593](https://avatars.discourse-cdn.com/v4/letter/r/8edcca/32.png) [@raj.shah1593](https://discuss.elastic.co/u/raj.shah1593)
#### Post date: [April 16, 2018, 12:54pm UTC](https://discuss.elastic.co/t/elastic-search-max-num-segments-in-forcemerge/127472/15 "2018-04-16T12:54:45Z")

</div>

Hello,

Thank you for the help. I completed the force\_merge operation on 1.3 TB index.  
The segment count is now 1 per shard.  
It took 3.5 hours to complete force\_merge process.

We did force\_merge 1 year after creating the index. Now, if we plan to do force\_merge every 6 months, will it take the same amount of time or it should be lesser?

Thanks in advance 🙂

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 14, 2018, 12:54pm UTC](https://discuss.elastic.co/t/elastic-search-max-num-segments-in-forcemerge/127472/16 "2018-05-14T12:54:55Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
