# ElasticSearch Segment merge not happening when deleted documents count is greater than 50%

**URL:** <https://discuss.elastic.co/t/elasticsearch-segment-merge-not-happening-when-deleted-documents-count-is-greater-than-50/277209>\
**Category:** Elasticsearch\
**Created:** [June 28, 2021, 11:25am UTC](https://discuss.elastic.co/t/elasticsearch-segment-merge-not-happening-when-deleted-documents-count-is-greater-than-50/277209 "2021-06-28T11:25:27Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vignesh\_Ramesh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vignesh_ramesh/32/90897_2.png) [@Vignesh\_Ramesh](https://discuss.elastic.co/u/Vignesh_Ramesh)\
**Post date:** [June 28, 2021, 11:25am UTC](https://discuss.elastic.co/t/elasticsearch-segment-merge-not-happening-when-deleted-documents-count-is-greater-than-50/277209/1 "2021-06-28T11:25:27Z")

</div>

**Elasticsearch version: 7.10.0**

I have an elasticsearch index with 8 shards in 8 different nodes and with a document count greater than 25 million documents(nested not included). It's an heavy update index. The document size grows over a period of time because of deleted documents. I did a search on this issue and read blogs like one below which tells a **segment will automatically be merged when the deleted docs count in that segment is greater than 50%.**

> [@Too many Deleted Docs](https://discuss.elastic.co/t/too-many-deleted-docs/84964/4):
>
> Hi @jpountz Thanks a lot for your response. Please let me know if my understanding is correct, increasing reclaim\_deletes\_weight will favor segments with more deleted docs from the documentation. For the existing segments that have already reached max\_merged\_segment , will they be affected by this setting also? The reason why I am worried about the deleted documents , is that they are taking up lot of space and I have read these awesome blogs [http://blog.mikemccandless.com/2011/02/visualizing-…](http://blog.mikemccandless.com/2011/02/visualizing-lucenes-segment-merges.html)

I did a **/\_segment** s for the index and found segments like the below

```auto
"segments": {
                            "_bbx": {
                                "generation": 14685,
                                "num_docs": 27901732,
                                "deleted_docs": 23290932,
                                "size_in_bytes": 5071187083,
                                "memory_in_bytes": 137008,
                                "committed": true,
                                "search": true,
                                "version": "8.7.0",
                                "compound": false,
                                "attributes": {
                                    "Lucene87StoredFieldsFormat.mode": "BEST_SPEED"
                                }
                            },

```

Full response of /\_segment call can be found here

> **[segments.txt](https://drive.google.com/file/d/1mLE2xw0u7lnogHnfzz65rWCBS8JrcnNm/view?usp=sharing)**
>
> Google Drive file.

In many **segments like the one above the deleted\_docs count is more than 75% of the num\_docs**  **but is still not getting merged**. We haven't set any max\_merged\_segment so the default is 5gb. We also haven't changed any MergePolicy and are using the default ones as of Es version 7.10.0.

Is my understanding correct ?

Any thoughts on this would be helpful. Thanks in advance.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [July 3, 2021, 5:54pm UTC](https://discuss.elastic.co/t/elasticsearch-segment-merge-not-happening-when-deleted-documents-count-is-greater-than-50/277209/2 "2021-07-03T17:54:47Z")

</div>

> [@Vignesh\_Ramesh](#):
>
> ```auto
> "num_docs": 27901732,
> "deleted_docs": 23290932,
> 
> ```

I believe you have 23,290,932 deleted documents out of a total of 51,192,664 (27,901,732 + 23,290,932) documents which means 45.5% are deleted in that segment.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 31, 2021, 5:55pm UTC](https://discuss.elastic.co/t/elasticsearch-segment-merge-not-happening-when-deleted-documents-count-is-greater-than-50/277209/3 "2021-07-31T17:55:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.