# ES 5.2 index size keeps growing with every indexing run

**URL:** <https://discuss.elastic.co/t/es-5-2-index-size-keeps-growing-with-every-indexing-run/105935>\
**Category:** Elasticsearch\
**Created:** [October 31, 2017, 5:43pm UTC](https://discuss.elastic.co/t/es-5-2-index-size-keeps-growing-with-every-indexing-run/105935 "2017-10-31T17:43:33Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![jigs007](https://avatars.discourse-cdn.com/v4/letter/j/ccd318/32.png) [@jigs007](https://discuss.elastic.co/u/jigs007)\
**Post date:** [October 31, 2017, 5:43pm UTC](https://discuss.elastic.co/t/es-5-2-index-size-keeps-growing-with-every-indexing-run/105935/1 "2017-10-31T17:43:34Z")

</div>

we have an issue with ES 5.2 (3 node cluster) where the index size keeps growing disproportionately. We run some background job everyday which indexes 5million existing cases in 3 hr window. we have refresh interval set to 1s as required by business. The update in documents is marginal. like 1% of total index size.

So far this has been working well with our old ES 1.3.4. we run same job everyday on 1.3.4 as well. During the run the index size grows and falls back to expected size due to segment merges. But in 5.2 this size never drops. we can see the merges are happening but for some reason the index size just grows.

Example:  
ES 1.3.4  
500k documents  
initial index size: 100GB after a job run 125gb, after sometime reduces back to 112gb. (2gb increase due new info on existing documents)

ES 5.2  
500k documents  
initial index size: 100GB after job run 125gb, size drop back by hardly 3-5gb. This means the index in 5.2 grows by 22gb everyday vs 2gb in 1.3.

Cluster spec:  
3 node, each node 2.5tb storage, 16gb heap, 2 master nodes, index has 5 primary shards, with replication factor 1, memory and OS swap disabled, refresh\_interval 1s. Running on 64 bit linux, jdk 1.8\_0\_20\_64bit

Also, have tried below settings to tune merging:  
"index.merge.scheduler.max\_merge\_count": 100,  
"index.merge.scheduler.max\_thread\_count":4,  
"index.merge.policy.max\_merged\_segment": "12gb",  
"index.merge.policy.reclaim\_deletes\_weight":3.0,  
"index.merge.policy.max\_merge\_at\_once":10,  
"index.merge.policy.segments\_per\_tier":10,  
"index.merge.policy.floor\_segment":"5mb"

Also tried \_forceMerge. which merges segments as expected freeing up deleted docs space.  
but results from "myindex/ \_segments" seems odd. Adding all segments "size\_in\_bytes" doesnt match the size of index??

i.e.  
If I have 5 segments then their sum of "size\_in\_bytes" is 10gb vs, the shard size under "\_cat/shards?v" shows 70gb? The docs count matches as expected though.

Also, checked the linux "top" command output to see the memory is locked for each node. it shows node1 has 146gb, node2 67gb, node3 has 35gb.

When we restart ES 5.2 cluster, the index size returns to expected 100gb initial state but then keeps growing with daily indexing run.

Have checked for possible doc\_values, we are using same fields in 1.3.4 and 5.2.

Pls help.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [November 4, 2017, 8:50am UTC](https://discuss.elastic.co/t/es-5-2-index-size-keeps-growing-with-every-indexing-run/105935/2 "2017-11-04T08:50:59Z")

</div>

What are the mappings?

---

<div class="post-metadata">

**Author:** ![jigs007](https://avatars.discourse-cdn.com/v4/letter/j/ccd318/32.png) [@jigs007](https://discuss.elastic.co/u/jigs007)\
**Post date:** [November 4, 2017, 11:51am UTC](https://discuss.elastic.co/t/es-5-2-index-size-keeps-growing-with-every-indexing-run/105935/3 "2017-11-04T11:51:26Z")

</div>

Hi @warkolm, here is the mapping Part 1: (Since it exceeds 7000 chars)  
\> {

> ```
> "myDocumentType": {
> "_all": {
> "enabled": false
> },
> "dynamic": "strict",
> "properties": {
> "field1": {
> "properties": {
> "field2": {
> "type": "object",
> "enabled": false
> },
> "field3": {
> "type": "object",
> "enabled": false
> },
> "field4": {
> "properties": {
> "field5": {
> "type": "text",
> "fielddata": true
> },
> "field6": {
> "type": "keyword"
> },
> "field7": {
> "type": "text",
> "fielddata": true
> },
> "field8": {
> "type": "object",
> "enabled": false
> }
> }
> },
> "field9": {
> "properties": {
> "field10": {
> "type": "object",
> "enabled": false
> },
> "field11": {
> "type": "keyword"
> },
> "field12": {
> "type": "keyword"
> },
> "field13": {
> "properties": {
> "field14": {
> "type": "object",
> "enabled": false
> },
> "field15": {
> "type": "object",
> "enabled": false
> },
> "field16": {
> "type": "object",
> "enabled": false
> },
> "field17": {
> "type": "object",
> "enabled": false
> }
> }
> },
> "field18": {
> "type": "object",
> "enabled": false
> }
> }
> },
> "field19": {
> "type": "keyword"
> },
> "field20": {
> "properties": {
> "field21": {
> "properties": {
> "field22": {
> "type": "long"
> },
> "field23": {
> "type": "long"
> },
> "field24": {
> "type": "long"
> },
> "field25": {
> "type": "long"
> },
> "field26": {
> "type": "long"
> }
> }
> },
> "field27": {
> "type": "long"
> }
> }
> },
> "field28": {
> "type": "object",
> "enabled": false
> },
> "field29": {
> "properties": {
> "field30": {
> "type": "object",
> "enabled": false
> },
> "field31": {
> "type": "object",
> "enabled": false
> },
> "field32": {
> "type": "object",
> "enabled": false
> },
> "field33": {
> "type": "long"
> }
> }
> },
> "field34": {
> "properties": {
> "field35": {
> "type": "keyword"
> },
> "field36": {
> "type": "text",
> "analyzer": "lowercase_keyword",
> "fielddata": true
> },
> "field37": {
> "type": "text",
> "analyzer": "lowercase_keyword",
> "fielddata": true
> }
> }
> },
> "field38": {
> "type": "keyword"
> },
> "field39": {
> "type": "text",
> "analyzer": "lowercase_keyword",
> "fielddata": true
> },
> "field40": {
> "type": "object",
> "enabled": false
> },
> "field41": {
> "type": "object",
> "enabled": false
> },
> "field42": {
> "properties": {
> "field43": {
> "type": "object",
> "enabled": false
> },
> "field44": {
> "type": "object",
> "enabled": false
> },
> "field45": {
> "type": "object",
> "enabled": false
> },
> "field46": {
> "type": "long"
> }
> }
> },
> 
> ```

---

<div class="post-metadata">

**Author:** ![jigs007](https://avatars.discourse-cdn.com/v4/letter/j/ccd318/32.png) [@jigs007](https://discuss.elastic.co/u/jigs007)\
**Post date:** [November 4, 2017, 11:52am UTC](https://discuss.elastic.co/t/es-5-2-index-size-keeps-growing-with-every-indexing-run/105935/4 "2017-11-04T11:52:30Z")

</div>

Part 2:

> ```
> "field47": {
> "properties": {
> "field48": {
> "type": "object",
> "enabled": false
> },
> "field49": {
> "type": "object",
> "enabled": false
> },
> "field50": {
> "type": "object",
> "enabled": false
> },
> "field51": {
> "properties": {
> "field52": {
> "type": "object",
> "enabled": false
> },
> "field53": {
> "type": "object",
> "enabled": false
> },
> "field54": {
> "type": "object",
> "enabled": false
> },
> "field55": {
> "type": "object",
> "enabled": false
> }
> }
> },
> "field56": {
> "type": "object",
> "enabled": false
> },
> "field57": {
> "type": "object",
> "enabled": false
> },
> "field58": {
> "type": "object",
> "enabled": false
> },
> "field59": {
> "type": "object",
> "enabled": false
> },
> "field60": {
> "type": "object",
> "enabled": false
> },
> "field61": {
> "type": "object",
> "enabled": false
> }
> }
> },
> "field62": {
> "properties": {
> "field63": {
> "type": "object",
> "enabled": false
> },
> "field64": {
> "type": "object",
> "enabled": false
> },
> "field65": {
> "type": "object",
> "enabled": false
> },
> "field66": {
> "type": "long"
> }
> }
> },
> "field67": {
> "type": "keyword"
> },
> "field68": {
> "properties": {
> "field69": {
> "type": "object",
> "enabled": false
> },
> "field70": {
> "type": "object",
> "enabled": false
> },
> "field71": {
> "type": "object",
> "enabled": false
> },
> "field72": {
> "type": "long",
> "index": false
> }
> }
> },
> "field73": {
> "properties": {
> "field74": {
> "type": "object",
> "enabled": false
> },
> "field75": {
> "type": "object",
> "enabled": false
> },
> "field76": {
> "type": "object",
> "enabled": false
> },
> "field77": {
> "type": "long",
> "index": false
> }
> }
> },
> "field78": {
> "properties": {
> "field79": {
> "type": "text",
> "fielddata": true
> },
> "field80": {
> "type": "keyword"
> },
> "field81": {
> "type": "text",
> "fielddata": true
> },
> "field82": {
> "type": "object",
> "enabled": false
> }
> }
> },
> "field83": {
> "type": "object",
> "enabled": false
> },
> "field84": {
> "properties": {
> "field85": {
> "type": "object",
> "enabled": false
> },
> "field86": {
> "type": "object",
> "enabled": false
> },
> "field87": {
> "type": "object",
> "enabled": false
> },
> "field88": {
> "type": "text",
> "analyzer": "lowercase_keyword",
> "field89": {
> "sort": {
> "type": "text",
> "analyzer": "ducet_sort",
> "fielddata": true
> }
> }
> },
> "field90": {
> "type": "object",
> "enabled": false
> },
> "field91": {
> "type": "keyword"
> },
> "field92": {
> "type": "object",
> "enabled": false
> },
> "field93": {
> "type": "object",
> "enabled": false
> },
> "field94": {
> "type": "object",
> "enabled": false
> },
> "field95": {
> "type": "object",
> "enabled": false
> },
> "field96": {
> "type": "object",
> "enabled": false
> },
> "field97": {
> "type": "object",
> "enabled": false
> }
> }
> },
> "field98": {
> "type": "object",
> "enabled": false
> },
> "field99": {
> "type": "object",
> "enabled": false
> },
> "field100": {
> "type": "object",
> "enabled": false
> },
> "field101": {
> "type":"object",
> "enabled": false
> },
> 
> ```

---

<div class="post-metadata">

**Author:** ![jigs007](https://avatars.discourse-cdn.com/v4/letter/j/ccd318/32.png) [@jigs007](https://discuss.elastic.co/u/jigs007)\
**Post date:** [November 4, 2017, 11:54am UTC](https://discuss.elastic.co/t/es-5-2-index-size-keeps-growing-with-every-indexing-run/105935/5 "2017-11-04T11:54:03Z")

</div>

Part3:

> ```
> "field101": {
> "type": "boolean"
> },
> "field102": {
> "properties": {
> "field103": {
> "type": "keyword"
> },
> "field104": {
> "type":"object",
> "enabled": false
> },
> "field105": {
> "type": "object",
> "enabled": false
> },
> "field106": {
> "type": "object",
> "enabled": false
> }
> }
> },
> "field107": {
> "type": "object",
> "enabled": false
> },
> "field108": {
> "type": "object",
> "enabled": false
> },
> "field109": {
> "type": "object",
> "enabled": false
> },
> "field110": {
> "type": "keyword"
> },
> "field111": {
> "type": "object",
> "enabled": false
> },
> "field112": {
> "type": "object",
> "enabled": false
> },
> "field113": {
> "type": "boolean",
> "index": false
> },
> "field114": {
> "type": "object",
> "enabled": false
> },
> "field115": {
> "type": "object",
> "enabled": false
> },
> "field116": {
> "type": "object",
> "enabled": false
> },
> "field117": {
> "type": "object",
> "enabled": false
> },
> "field118": {
> "type": "object",
> "enabled": false
> },
> "field119": {
> "type": "object",
> "enabled": false
> },
> "field120": {
> "type": "long"
> }
> }
> },
> "id": {
> "type": "keyword"
> },
> "version": {
> "type": "object",
> "enabled": false
> }
> }} }
> 
> ```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 2, 2017, 11:54am UTC](https://discuss.elastic.co/t/es-5-2-index-size-keeps-growing-with-every-indexing-run/105935/6 "2017-12-02T11:54:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.