# Elasticsearch 1.7.5.1: delete document has no impact on the index size

**URL:** <https://discuss.elastic.co/t/elasticsearch-1-7-5-1-delete-document-has-no-impact-on-the-index-size/77533>\
**Category:** Elasticsearch\
**Created:** [March 6, 2017, 3:44pm UTC](https://discuss.elastic.co/t/elasticsearch-1-7-5-1-delete-document-has-no-impact-on-the-index-size/77533 "2017-03-06T15:44:27Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Antoine\_Brun](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/antoine_brun/32/3774_2.png) [@Antoine\_Brun](https://discuss.elastic.co/u/Antoine_Brun)\
**Post date:** [March 6, 2017, 3:44pm UTC](https://discuss.elastic.co/t/elasticsearch-1-7-5-1-delete-document-has-no-impact-on-the-index-size/77533/1 "2017-03-06T15:44:27Z")

</div>

Hello,

we have a ES cluster 1.7 (it's production so we couldn' t upgrade it to a more recent version) and we are storing network device syslogs in daily indexes.  
Because the volume of data collected is huge, we have scripts to manage the log retention policy: we keep security logs for 90 days but traffic logs (99% of the data) are deleted after 8 days.  
To delete we use this kind of query:  
2017-03-06 15:36:03: curl -XDELETE '127.0.0.1:9200/prodlogs-2017-02-27/logs/\_query' -d '{  
"query": {  
"bool": {  
"must": [{  
"query\_string": {  
"query": "type:traffic",  
"default\_operator": "AND"  
}  
}, {  
"range": {  
"date": {  
"lt": "now-8d"  
}  
}  
}]  
}  
}  
}'

so every day we delete 98-99% of the data in the index and the kopf plugin show that the number of document is reduced as expected.  
The problem we have is that the size of the index is no decreased.

I have tried to optimize the index but the size stays the same.

This is a big problem for us because we are running out of disk space (4Tb).

Any idea here would be more that welcome

Antoine

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [March 6, 2017, 3:50pm UTC](https://discuss.elastic.co/t/elasticsearch-1-7-5-1-delete-document-has-no-impact-on-the-index-size/77533/2 "2017-03-06T15:50:32Z")

</div>

I would recommend storing your two different types of data in two separate indices as they have different retention periods. This would allow you to simply delete complete daily indices rather than using delete by query. For the data that is in the current indices holding all types of data, it is probably easier and more efficient to reindex the security logs into separate indices as the volumes are low and then delete the large indices that contain traffic data.

---

<div class="post-metadata">

**Author:** ![Antoine\_Brun](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/antoine_brun/32/3774_2.png) [@Antoine\_Brun](https://discuss.elastic.co/u/Antoine_Brun)\
**Post date:** [March 6, 2017, 4:20pm UTC](https://discuss.elastic.co/t/elasticsearch-1-7-5-1-delete-document-has-no-impact-on-the-index-size/77533/3 "2017-03-06T16:20:33Z")

</div>

yes, that's a possibility we'll have to consider.  
In the mean time, I just found out that I can use the option only\_expunge\_deletes when optimizing

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 3, 2017, 4:20pm UTC](https://discuss.elastic.co/t/elasticsearch-1-7-5-1-delete-document-has-no-impact-on-the-index-size/77533/4 "2017-04-03T16:20:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.