# Bulk Indexing - Tier Merge Policy

**URL:** https://discuss.elastic.co/t/bulk-indexing-tier-merge-policy/6161
**Category:** Elasticsearch
**Created:** [December 15, 2011, 1:21am UTC](https://discuss.elastic.co/t/bulk-indexing-tier-merge-policy/6161 "2011-12-15T01:21:11Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![phobos182](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/phobos182/32/3011_2.png) [@phobos182](https://discuss.elastic.co/u/phobos182)
#### Post date: [December 15, 2011, 1:21am UTC](https://discuss.elastic.co/t/bulk-indexing-tier-merge-policy/6161/1 "2011-12-15T01:21:11Z")

</div>

Quick question. Is it best to increase the number of segments for the Tier Merge Policy to decrease the time spent merging during a bulk indexing? My reasoning being after the bulk indexing has been complete, and optimize can be issued, and dynamically changing the segments\_per\_tier back to something like 5ish. I want to minimize the number of segments to decrease query speed once an index has been bulk indexed.

What are some other settings individuals are using for Bulk indexing to get it done quickly, and then change it to something that creates smaller segments for quicker searching?

---

<div class="post-metadata">

### Author: ![Karussell1](https://avatars.discourse-cdn.com/v4/letter/k/50afbb/32.png) [@Karussell1](https://discuss.elastic.co/u/Karussell1)
#### Post date: [December 15, 2011, 8:18am UTC](https://discuss.elastic.co/t/bulk-indexing-tier-merge-policy/6161/2 "2011-12-15T08:18:57Z")

</div>

You can read about bulk indexing tuning here

> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.

> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.

and yes the merge.policy.merge\_factor can be dynamically updated to a  
higher value (\>20) to improve indexing speed. Afterwards optimize.  
(but you'll have to check if overall execution time is really  
smaller).

Peter.

On 15 Dez., 02:21, phobos182 [phobos...@gmail.com](mailto:phobos...@gmail.com) wrote:

> Quick question. Is it best to increase the number of segments for the Tier  
> Merge Policy to decrease the time spent merging during a bulk indexing? My  
> reasoning being after the bulk indexing has been complete, and optimize can  
> be issued, and dynamically changing the segments\_per\_tier back to something  
> like 5ish. I want to minimize the number of segments to decrease query speed  
> once an index has been bulk indexed.
> 
> What are some other settings individuals are using for Bulk indexing to get  
> it done quickly, and then change it to something that creates smaller  
> segments for quicker searching?
> 
> --  
> View this message in context:[http://elasticsearch-users.115913.n3.nabble.com/Bulk-Indexing-Tier-Me](http://elasticsearch-users.115913.n3.nabble.com/Bulk-Indexing-Tier-Me)...  
> Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com).

---

<div class="post-metadata">

### Author: ![phobos182](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/phobos182/32/3011_2.png) [@phobos182](https://discuss.elastic.co/u/phobos182)
#### Post date: [December 15, 2011, 4:27pm UTC](https://discuss.elastic.co/t/bulk-indexing-tier-merge-policy/6161/3 "2011-12-15T16:27:46Z")

</div>

Merge Factor does not work with the Tiered Merge Policy. They have a high watermark for segments, and try to merge down to get under the watermark. My question is how to index fast, but then optimize after the fact for a one time bulk index. How are individuals accomplishing this themselves? If I have 500 million documents to index, I don't want 2 segments total, I want something like 20-30. But after it's done, the searching is so slow I want to get to 5-10 segments to increase speed.

---

<div class="post-metadata">

### Author: ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [December 16, 2011, 3:42pm UTC](https://discuss.elastic.co/t/bulk-indexing-tier-merge-policy/6161/4 "2011-12-16T15:42:20Z")

</div>

Yes, increasing the segments should help for bulk indexing. Also, another  
one that many people miss is to simply start with no replicas, and increase  
the replica count once indexing is done.

On Thu, Dec 15, 2011 at 6:27 PM, phobos182 [phobos182@gmail.com](mailto:phobos182@gmail.com) wrote:

> Merge Factor does not work with the Tiered Merge Policy. They have a high  
> watermark for segments, and try to merge down to get under the watermark.  
> My  
> question is how to index fast, but then optimize after the fact for a one  
> time bulk index. How are individuals accomplishing this themselves? If I  
> have 500 million documents to index, I don't want 2 segments total, I want  
> something like 20-30. But after it's done, the searching is so slow I want  
> to get to 5-10 segments to increase speed.
> 
> --  
> View this message in context:  
> [http://elasticsearch-users.115913.n3.nabble.com/Bulk-Indexing-Tier-Merge-Policy-tp3587182p3589130.html](http://elasticsearch-users.115913.n3.nabble.com/Bulk-Indexing-Tier-Merge-Policy-tp3587182p3589130.html)  
> Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com).

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 3:45am UTC](https://discuss.elastic.co/t/bulk-indexing-tier-merge-policy/6161/5 "2017-07-06T03:45:12Z")

</div>


