# Expunge Deleted Docs from index **URL:** \ **Category:** Elasticsearch\ **Created:** [June 18, 2013, 11:02am UTC](https://discuss.elastic.co/t/expunge-deleted-docs-from-index/12458 "2013-06-18T11:02:30Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![tarang\_dawer](https://avatars.discourse-cdn.com/v4/letter/t/c0e974/32.png) [@tarang\_dawer](https://discuss.elastic.co/u/tarang_dawer)\ **Post date:** [June 18, 2013, 11:02am UTC](https://discuss.elastic.co/t/expunge-deleted-docs-from-index/12458/1 "2013-06-18T11:02:30Z")
Hi I have deleted a mapping from my index. ES , marks the deleted documents in the index as logically deleted, but physical space is relcaimed only by explicitly optimizing with expunge\_deletes=true. Does ES optimize an index , which has 50 percent Deletions(more that tiered merge policy's merge.policy.expunge\_deletes\_allowed 's Default 10 percent , even when there is no indexing activity on the index ? Also , would that activity be as resource intensive as explicit optimization ? or like a regular indexing -time merge operation (which doesnot seems to eat up much resources ) . If it's possible , then how to configure it ? explicit optimization is a indeed a heavy process , and thus i m keen to avoid it. By issuing a explicit optimization call after the delete , would bring by queries to a dead halt. Also , re-indexing is not possible because of huge data size. ps: i am using ES 0.90.1 . Please Help Thanks in Advance. Regards Tarang Dawer -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jpountz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jpountz/32/45836_2.png) [@jpountz](https://discuss.elastic.co/u/jpountz)\ **Post date:** [June 18, 2013, 1:43pm UTC](https://discuss.elastic.co/t/expunge-deleted-docs-from-index/12458/2 "2013-06-18T13:43:01Z")
Hi, Physical space is also reclaimed through background merges, which are triggered when you index documents. The difference is that the optimize API allows you to be more aggressive regarding the maximum number of segments you want in your index (the smaller the number of segments, the more expensive the operation). Although Elasticsearch might not look very aggressive at reclaiming space, it is important to not trigger merge operations too often because merging can require a lot of I/O which might slow down searching. If space is still more important than searching speed, you could try to increase index.reclaim\_deletes\_weight, but I wouldn't recommend to trigger merges explicitely from the application when deleting documents. -- Adrien Grand -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![tarang\_dawer](https://avatars.discourse-cdn.com/v4/letter/t/c0e974/32.png) [@tarang\_dawer](https://discuss.elastic.co/u/tarang_dawer)\ **Post date:** [June 19, 2013, 7:40am UTC](https://discuss.elastic.co/t/expunge-deleted-docs-from-index/12458/3 "2013-06-19T07:40:40Z")
Hi Adrien Thanks for your response. So , it seems to me that , for an index in which indexing is a ongoing operation , it's a viable option to configure it's settings with index.reclaim\_deletes\_weight 's value a bit greater , to influence the segment selection policy , and thus to reclaim the physical space back. But if the index has no more indexing activity ,since there would not be any more merges due to the lack of indexing activity , an explicit optimization request would then be required to reclaim the physical space . Is my understanding correct ? Regards Tarang Dawer On Tue, Jun 18, 2013 at 7:13 PM, Adrien Grand \< [adrien.grand@elasticsearch.com](mailto:adrien.grand@elasticsearch.com)\> wrote: > Hi, > > Physical space is also reclaimed through background merges, which are > triggered when you index documents. The difference is that the optimize API > allows you to be more aggressive regarding the maximum number of segments > you want in your index (the smaller the number of segments, the more > expensive the operation). > > Although Elasticsearch might not look very aggressive at reclaiming space, > it is important to not trigger merge operations too often because merging > can require a lot of I/O which might slow down searching. If space is still > more important than searching speed, you could try to increase > index.reclaim\_deletes\_weight, but I wouldn't recommend to trigger merges > explicitely from the application when deleting documents. > > -- > Adrien Grand > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jpountz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jpountz/32/45836_2.png) [@jpountz](https://discuss.elastic.co/u/jpountz)\ **Post date:** [June 19, 2013, 8:26am UTC](https://discuss.elastic.co/t/expunge-deleted-docs-from-index/12458/4 "2013-06-19T08:26:45Z")
On Wed, Jun 19, 2013 at 9:40 AM, Tarang Dawer [tarang.dawer@gmail.com](mailto:tarang.dawer@gmail.com)wrote: > So , it seems to me that , for an index in which indexing is a ongoing > operation , it's a viable option to configure it's settings with index.reclaim\_deletes\_weight > 's value a bit greater , to influence the segment selection policy , and > thus to reclaim the physical space back. > > But if the index has no more indexing activity ,since there would not be > any more merges due to the lack of indexing activity , an explicit > optimization request would then be required to reclaim the physical space . > > Is my understanding correct ? This is correct. -- Adrien Grand -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![tarang\_dawer](https://avatars.discourse-cdn.com/v4/letter/t/c0e974/32.png) [@tarang\_dawer](https://discuss.elastic.co/u/tarang_dawer)\ **Post date:** [June 19, 2013, 1:57pm UTC](https://discuss.elastic.co/t/expunge-deleted-docs-from-index/12458/5 "2013-06-19T13:57:36Z")
Thanks Adrien. Could you please clarify 1 more thing ? If i have 6 indices in my cluster , 3 of them having deletes in them , and rest do not, then 1 .Optimization with expunge deletes only , will it only optimize Indices with deletes , and leave the rest untouched ? 1. Optimization (normal , without expunge deletes only option) , will effect all the 6 indices , merging the existing segments again (thus causing unnecessary heavy I/O (because of untouched 3 indices )) ? Regards Tarang Dawer On Wed, Jun 19, 2013 at 1:56 PM, Adrien Grand \< [adrien.grand@elasticsearch.com](mailto:adrien.grand@elasticsearch.com)\> wrote: > On Wed, Jun 19, 2013 at 9:40 AM, Tarang Dawer [tarang.dawer@gmail.com](mailto:tarang.dawer@gmail.com)wrote: > > > So , it seems to me that , for an index in which indexing is a ongoing > > operation , it's a viable option to configure it's settings with index.reclaim\_deletes\_weight > > 's value a bit greater , to influence the segment selection policy , and > > thus to reclaim the physical space back. > > > > But if the index has no more indexing activity ,since there would not be > > any more merges due to the lack of indexing activity , an explicit > > optimization request would then be required to reclaim the physical space . > > > > Is my understanding correct ? > > This is correct. > > -- > Adrien Grand > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:30am UTC](https://discuss.elastic.co/t/expunge-deleted-docs-from-index/12458/6 "2017-07-06T02:30:22Z")