# How does segment merging work

**URL:** <https://discuss.elastic.co/t/how-does-segment-merging-work/26416>\
**Category:** Elasticsearch\
**Created:** [July 28, 2015, 2:00pm UTC](https://discuss.elastic.co/t/how-does-segment-merging-work/26416 "2015-07-28T14:00:23Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![javadevmtl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/javadevmtl/32/45613_2.png) [@javadevmtl](https://discuss.elastic.co/u/javadevmtl)\
**Post date:** [July 28, 2015, 2:00pm UTC](https://discuss.elastic.co/t/how-does-segment-merging-work/26416/1 "2015-07-28T14:00:23Z")

</div>

So i read the docs, but just curious...

Using ES 1.6.0

I had an index with: 257,947,981 documents and it was 1.28TB  
Then I did some bulk updates to that index which ended up being: 257,997,034 docs and 1.56TB

Then for sh\*ts and giggles I ran POST /my-index/\_optimize?max\_num\_segments=1

Now running the optimize API the index size has grown to 1.7TB  
Each shard is about 60 segments each except one is at 40.

Will this really reduces all shards to 1 segments or best it can?  
Also will this eventually end up saving space?  
And how long should I expect this to execute?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 28, 2015, 11:32pm UTC](https://discuss.elastic.co/t/how-does-segment-merging-work/26416/2 "2015-07-28T23:32:21Z")

</div>

An index won't grow in size if you optimise, it removes any deleted docs and then merges the segments. I'd say there was more indexing happening at the same time.

An optimise will take as long as it takes, it's dependant on the size of the index, load on the system, IO speeds etc etc.

---

<div class="post-metadata">

**Author:** ![javadevmtl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/javadevmtl/32/45613_2.png) [@javadevmtl](https://discuss.elastic.co/u/javadevmtl)\
**Post date:** [July 29, 2015, 1:43pm UTC](https://discuss.elastic.co/t/how-does-segment-merging-work/26416/3 "2015-07-29T13:43:15Z")

</div>

I did the test. I only optimized after bulk updates finished.

Index at start was 1.28TB  
Index after updates: 1.56TB  
Index while optimizing expanded to 2+TB  
Index after optimizing finished: 1.25TB

Now running optimize on none updated index. I.e: No new inserts or updates.

---

<div class="post-metadata">

**Author:** ![javadevmtl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/javadevmtl/32/45613_2.png) [@javadevmtl](https://discuss.elastic.co/u/javadevmtl)\
**Post date:** [July 29, 2015, 2:11pm UTC](https://discuss.elastic.co/t/how-does-segment-merging-work/26416/4 "2015-07-29T14:11:56Z")

</div>

Yep even on a non updated index there's expansion going on.

It started at 1.30TB  
Now during optimization it's up to 1.37TB and growing...

I guess it expands when the new segments is created so the old one can get merged into it?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 29, 2015, 9:22pm UTC](https://discuss.elastic.co/t/how-does-segment-merging-work/26416/5 "2015-07-29T21:22:04Z")

</div>

Initially, yes, but it will drop back down.

---

<div class="post-metadata">

**Author:** ![javadevmtl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/javadevmtl/32/45613_2.png) [@javadevmtl](https://discuss.elastic.co/u/javadevmtl)\
**Post date:** [July 30, 2015, 2:11am UTC](https://discuss.elastic.co/t/how-does-segment-merging-work/26416/6 "2015-07-30T02:11:55Z")

</div>

Cool, I can confirm it. During optimization it expands and then drops back to original size about slightly less.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:58pm UTC](https://discuss.elastic.co/t/how-does-segment-merging-work/26416/7 "2017-07-05T23:58:12Z")

</div>


