# Index size is increased while deleting

**URL:** <https://discuss.elastic.co/t/index-size-is-increased-while-deleting/217804>\
**Category:** Elasticsearch\
**Created:** [February 4, 2020, 12:38pm UTC](https://discuss.elastic.co/t/index-size-is-increased-while-deleting/217804 "2020-02-04T12:38:48Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![nhatnam](https://avatars.discourse-cdn.com/v4/letter/n/848f3c/32.png) [@nhatnam](https://discuss.elastic.co/u/nhatnam)\
**Post date:** [February 4, 2020, 12:38pm UTC](https://discuss.elastic.co/t/index-size-is-increased-while-deleting/217804/1 "2020-02-04T12:38:48Z")

</div>

I've just upgraded my cluster from v5.3 to v7.5 and I see strange behaviour.

I'm using Bulk API to delete and index docs to a newly created index. I'm sure that at the beginning my bulk requests only contain deletion of non-existent docs but I see the number of deleted docs (docs.deleted) and the index size (store.size\_in\_bytes) keep going up.

I don't see this behaviour in ES 5.3 cluster.

Can someone shed some light on this please?

I'm running the cluster with ~20 nodes and 6 shards, each shard has 20 replicas. Below you can find my cluster & index settings.

> {  
> "settings" : {  
> "analysis": {  
> "analyzer": {  
> "standard\_lowercase\_analyzer": {  
> "tokenizer": "standard",  
> "filter": [  
> "lowercase",  
> "asciifolding"  
> ]  
> }  
> },  
> "normalizer": {  
> "uppercase\_normalizer": {  
> "type": "custom",  
> "filter": ["uppercase"]  
> },  
> "lowercase\_normalizer": {  
> "type": "custom",  
> "filter": ["lowercase"]  
> }  
> }  
> },  
> "refresh\_interval": -1,  
> "number\_of\_shards": 6,  
> "auto\_expand\_replicas": false,  
> "search": {  
> "slowlog": {  
> "threshold": {  
> "fetch": {  
> "warn": "1s",  
> "trace": "200ms",  
> "debug": "500ms",  
> "info": "800ms"  
> },  
> "query": {  
> "warn": "5s",  
> "trace": "200ms",  
> "debug": "400ms",  
> "info": "1s"  
> }  
> }  
> }  
> },  
> "queries": {  
> "cache": {  
> "enabled": "true"  
> }  
> }  
> }  
> }

> cluster.name: classified-search  
> node.name: ${HOSTNAME}  
> plugin.mandatory: discovery-ec2,analysis-icu
> 
> network.bind\_host: 0.0.0.0  
> network.publish\_host: 0.0.0.0  
> network.host: _ec2:privateIp_
> 
> xpack.security.enabled: true  
> xpack.security.transport.ssl.enabled: true  
> xpack.security.transport.ssl.verification\_mode: certificate  
> xpack.security.transport.ssl.keystore.path: elastic-certificates.p12  
> xpack.security.transport.ssl.truststore.path: elastic-certificates.p12  
> xpack.security.authc:  
> anonymous:  
> roles: monitoring\_user  
> authz\_exception: true
> 
> discovery.seed\_providers: ec2  
> discovery.ec2.endpoint: [ec2.eu-west-1.amazonaws.com](http://ec2.eu-west-1.amazonaws.com)  
> discovery.ec2.host\_type: private\_ip  
> discovery.ec2.availability\_zones: eu-west-1a,eu-west-1b,eu-west-1c  
> discovery.ec2.tag.Environment: ENVIRONMENT
> 
> path:  
> data: /media/ephemeral0  
> logs: /var/log/elasticsearch
> 
> http.cors.enabled: true  
> http.cors.allow-origin: /https?://localhost(:[0-9]+)?/
> 
> indices.queries.cache.size: 20%  
> indices.requests.cache.size: 20%
> 
> action.auto\_create\_index: .watches,.triggered\_watches,.watcher-history-_,.monitoring-_,logstash\*,performance\*,-\*

---

<div class="post-metadata">

**Author:** ![Armin\_Braun](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/armin_braun/32/20092_2.png) [@Armin\_Braun](https://discuss.elastic.co/u/Armin_Braun)\
**Post date:** [February 12, 2020, 6:03am UTC](https://discuss.elastic.co/t/index-size-is-increased-while-deleting/217804/2 "2020-02-12T06:03:19Z")

</div>

Hi @nhatnam

the reason that deleting documents at least temporarily increases the size of an index is that when Lucene, the underlying storage engine ES uses, deletes a document it only adds the information that this document is deleted to the index. A delete does not initially cause a document to be removed from disk physically. You can find some detailed information on how this mechanism works under the hood in this older but still valid [article](https://www.elastic.co/blog/lucenes-handling-of-deleted-documents).

As explained in that article ES will eventually reclaim the disk space used by those deleted documents during segment merges. Also, if you are done writing to a certain index and know that you won't write to it again, you may force the reclaiming of disk space via the [force merge API](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-forcemerge.html).

As for

> I don't see this behaviour in ES 5.3 cluster.

I think this is likely a coincidence to some degree. ES 5.3 has the same behaviour in general but the actual specifics of how quickly disk space is reclaimed will vary according to the size of your indices, their settings, the Lucene/ES version etc. and is not easy to predict in the concrete case I think.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 11, 2020, 6:03am UTC](https://discuss.elastic.co/t/index-size-is-increased-while-deleting/217804/3 "2020-03-11T06:03:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.