# Optimize is blowing up index size **URL:** \ **Category:** Elasticsearch\ **Created:** [November 13, 2017, 4:22pm UTC](https://discuss.elastic.co/t/optimize-is-blowing-up-index-size/107432 "2017-11-13T16:22:10Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![morphers82](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/morphers82/32/22904_2.png) [@morphers82](https://discuss.elastic.co/u/morphers82)\ **Post date:** [November 13, 2017, 4:22pm UTC](https://discuss.elastic.co/t/optimize-is-blowing-up-index-size/107432/1 "2017-11-13T16:22:10Z")
So i'm on 1.7.x and i did an optimize on a large 10's of TBs index (haven't done one in a while) and all went well, the index size went down. I am currently optimizing another, however, the size has gone up amost 25% so far, with no signs of slowing down. what could be the issue? both had setting for only\_expunge\_deletes=true, first index has fewer writes too it, wile this second one is updated constantly. first index documents counts ie 5,000 (9,999) went down pretty quick ie 5,000(9,999) to 5,000 (6,999) second one, the doc counts are not really budging at all, especially for deleted counts (within parenthesis) I am in planning already to move everything to 6.0, but this is an immediate problem on 1.7.x ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [November 13, 2017, 4:35pm UTC](https://discuss.elastic.co/t/optimize-is-blowing-up-index-size/107432/2 "2017-11-13T16:35:37Z")
As segments merge, extra space will initially be consumed. It should free back up again as the source segments are marked for deletion, and then deleted (after successful re-indexing to larger segment). It could take a long time for this to happen, though. ---
**Author:** ![morphers82](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/morphers82/32/22904_2.png) [@morphers82](https://discuss.elastic.co/u/morphers82)\ **Post date:** [November 13, 2017, 4:47pm UTC](https://discuss.elastic.co/t/optimize-is-blowing-up-index-size/107432/3 "2017-11-13T16:47:39Z")
i'm now worried i'll run out of disk space before this can happen. ---
**Author:** ![morphers82](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/morphers82/32/22904_2.png) [@morphers82](https://discuss.elastic.co/u/morphers82)\ **Post date:** [November 13, 2017, 4:57pm UTC](https://discuss.elastic.co/t/optimize-is-blowing-up-index-size/107432/4 "2017-11-13T16:57:08Z")
I found this page [https://www.elastic.co/guide/en/elasticsearch/reference/1.7/index-modules-merge.html](https://www.elastic.co/guide/en/elasticsearch/reference/1.7/index-modules-merge.html) i lowered the defaults of index.merge.policy.max\_merge\_at\_once\_explicit and index.merge.policy.max\_merge\_at\_once . The size is not blowing up as fast, and actually went down for a minute, so I am hopeful. ---
**Author:** ![morphers82](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/morphers82/32/22904_2.png) [@morphers82](https://discuss.elastic.co/u/morphers82)\ **Post date:** [November 13, 2017, 5:02pm UTC](https://discuss.elastic.co/t/optimize-is-blowing-up-index-size/107432/5 "2017-11-13T17:02:56Z")
now I am wondering whether sparce doc values is the culprit, there are maybe 5% (guessing) of docs with some fields that have kbs of data in a few fields of each docs, while other docs those same fields were empty. i see that lucene doc value files are getting big. If i have to terminate the merge, will this stop the size explosion? how do i stop the optimize? ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [November 13, 2017, 5:04pm UTC](https://discuss.elastic.co/t/optimize-is-blowing-up-index-size/107432/6 "2017-11-13T17:04:17Z")
I don't think you can stop the optimize. There are no API calls to do so. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 11, 2017, 5:04pm UTC](https://discuss.elastic.co/t/optimize-is-blowing-up-index-size/107432/7 "2017-12-11T17:04:22Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.