# The documents deleted while the size of the index kremains the same after using delete by query? **URL:** \ **Category:** Elasticsearch\ **Created:** [July 23, 2013, 9:29am UTC](https://discuss.elastic.co/t/the-documents-deleted-while-the-size-of-the-index-kremains-the-same-after-using-delete-by-query/12893 "2013-07-23T09:29:23Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Jingang\_Wang](https://avatars.discourse-cdn.com/v4/letter/j/ce7236/32.png) [@Jingang\_Wang](https://discuss.elastic.co/u/Jingang_Wang)\ **Post date:** [July 23, 2013, 9:29am UTC](https://discuss.elastic.co/t/the-documents-deleted-while-the-size-of-the-index-kremains-the-same-after-using-delete-by-query/12893/1 "2013-07-23T09:29:23Z")
Dear all, I found that after I conducted a delete by query operation, the matched documents in the index have been deleted, while the size of the index remains the same. Could anybody tell me why? Thanks in advance! -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [July 23, 2013, 9:59am UTC](https://discuss.elastic.co/t/the-documents-deleted-while-the-size-of-the-index-kremains-the-same-after-using-delete-by-query/12893/2 "2013-07-23T09:59:34Z")
Hey, physical deletions only occur after a lucene merge, which is not executed immediately. Until then a document is only marked as deleted, but not wiped off the disk. If you put the same document twice, your disk space will increase as well until the next merge happens. This lies in the nature of the append-only mechanism of lucene. You can use the segment spy plugin to visualize merges. Another good read is this blog post > **[Visualizing Lucene's segment merges](https://blog.mikemccandless.com/2011/02/visualizing-lucenes-segment-merges.html)** > > If you've ever wondered how Lucene picks segments to merge during indexing, it looks something like this: That video displays segment merges... Hope this helps... --Alex On Tue, Jul 23, 2013 at 11:29 AM, Jingang Wang [bitwjg@gmail.com](mailto:bitwjg@gmail.com) wrote: > Dear all, > > I found that after I conducted a delete by query operation, the matched > documents in the index have been deleted, while the size of the index > remains the same. > > Could anybody tell me why? Thanks in advance! > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Jingang\_Wang](https://avatars.discourse-cdn.com/v4/letter/j/ce7236/32.png) [@Jingang\_Wang](https://discuss.elastic.co/u/Jingang_Wang)\ **Post date:** [July 24, 2013, 2:34am UTC](https://discuss.elastic.co/t/the-documents-deleted-while-the-size-of-the-index-kremains-the-same-after-using-delete-by-query/12893/3 "2013-07-24T02:34:28Z")
Hi Alex, Thanks for your informative reply. From the post, I have learned that the size of the index would remain the same until the new merge operation. If the index has been constructed over and then some documents in it were deleted, the size would not change because of no further merge execution. Is there some api could make the merge happens and decrease the size of the index? Thanks! On Tue, Jul 23, 2013 at 5:59 PM, Alexander Reelsen [alr@spinscale.de](mailto:alr@spinscale.de) wrote: > Hey, > > physical deletions only occur after a lucene merge, which is not executed > immediately. Until then a document is only marked as deleted, but not wiped > off the disk. If you put the same document twice, your disk space will > increase as well until the next merge happens. This lies in the nature of > the append-only mechanism of lucene. > > You can use the segment spy plugin to visualize merges. Another good read > is this blog post > > [Changing Bits: Visualizing Lucene's segment merges](http://blog.mikemccandless.com/2011/02/visualizing-lucenes-segment-merges.html) > > Hope this helps... > > --Alex > > On Tue, Jul 23, 2013 at 11:29 AM, Jingang Wang [bitwjg@gmail.com](mailto:bitwjg@gmail.com) wrote: > > > Dear all, > > > > I found that after I conducted a delete by query operation, the matched > > documents in the index have been deleted, while the size of the index > > remains the same. > > > > Could anybody tell me why? Thanks in advance! > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/kg4EwfzZZzE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/kg4EwfzZZzE/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- Wang Jingang(ηŽ‹ι‡‘εˆš) Ph.D. Candidate at Lab of High Volume Language Information Processing & Cloud Computing School of Computer Science Beijing Institute of Technology Beijing 100081 P.R China -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [July 24, 2013, 4:53am UTC](https://discuss.elastic.co/t/the-documents-deleted-while-the-size-of-the-index-kremains-the-same-after-using-delete-by-query/12893/4 "2013-07-24T04:53:36Z")
See Optimize API: [Elasticsearch Platform β€” Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-optimize/) -- David πŸ˜‰ Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 24 juil. 2013 Γ  04:34, Jingang Wang [bitwjg@gmail.com](mailto:bitwjg@gmail.com) a Γ©crit : Hi Alex, Thanks for your informative reply. From the post, I have learned that the size of the index would remain the same until the new merge operation. If the index has been constructed over and then some documents in it were deleted, the size would not change because of no further merge execution. Is there some api could make the merge happens and decrease the size of the index? Thanks! On Tue, Jul 23, 2013 at 5:59 PM, Alexander Reelsen [alr@spinscale.de](mailto:alr@spinscale.de) wrote: > Hey, > > physical deletions only occur after a lucene merge, which is not executed immediately. Until then a document is only marked as deleted, but not wiped off the disk. If you put the same document twice, your disk space will increase as well until the next merge happens. This lies in the nature of the append-only mechanism of lucene. > > You can use the segment spy plugin to visualize merges. Another good read is this blog post > [Changing Bits: Visualizing Lucene's segment merges](http://blog.mikemccandless.com/2011/02/visualizing-lucenes-segment-merges.html) > > Hope this helps... > > --Alex > > On Tue, Jul 23, 2013 at 11:29 AM, Jingang Wang [bitwjg@gmail.com](mailto:bitwjg@gmail.com) wrote: > > > Dear all, > > > > I found that after I conducted a delete by query operation, the matched documents in the index have been deleted, while the size of the index remains the same. > > > > ## Could anybody tell me why? Thanks in advance! > > > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit [https://groups.google.com/d/topic/elasticsearch/kg4EwfzZZzE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/kg4EwfzZZzE/unsubscribe). > To unsubscribe from this group and all its topics, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- Wang Jingang(ηŽ‹ι‡‘εˆš) Ph.D. Candidate at Lab of High Volume Language Information Processing & Cloud Computing School of Computer Science Beijing Institute of Technology Beijing 100081 P.R China -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Jingang\_Wang](https://avatars.discourse-cdn.com/v4/letter/j/ce7236/32.png) [@Jingang\_Wang](https://discuss.elastic.co/u/Jingang_Wang)\ **Post date:** [July 24, 2013, 5:53am UTC](https://discuss.elastic.co/t/the-documents-deleted-while-the-size-of-the-index-kremains-the-same-after-using-delete-by-query/12893/5 "2013-07-24T05:53:12Z")
Hi David, That's what I need, thank you!!! On Wed, Jul 24, 2013 at 12:53 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > See Optimize API: > [Elasticsearch Platform β€” Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-optimize/) > > -- > David πŸ˜‰ > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 24 juil. 2013 Γ  04:34, Jingang Wang [bitwjg@gmail.com](mailto:bitwjg@gmail.com) a Γ©crit : > > Hi Alex, > > Thanks for your informative reply. > > From the post, I have learned that the size of the index would remain the > same until the new merge operation. > If the index has been constructed over and then some documents in it were > deleted, the size would not change because of no further merge execution. > Is there some api could make the merge happens and decrease the size of > the index? > Thanks! > > On Tue, Jul 23, 2013 at 5:59 PM, Alexander Reelsen [alr@spinscale.de](mailto:alr@spinscale.de)wrote: > > > Hey, > > > > physical deletions only occur after a lucene merge, which is not executed > > immediately. Until then a document is only marked as deleted, but not wiped > > off the disk. If you put the same document twice, your disk space will > > increase as well until the next merge happens. This lies in the nature of > > the append-only mechanism of lucene. > > > > You can use the segment spy plugin to visualize merges. Another good read > > is this blog post > > > > [Changing Bits: Visualizing Lucene's segment merges](http://blog.mikemccandless.com/2011/02/visualizing-lucenes-segment-merges.html) > > > > Hope this helps... > > > > --Alex > > > > On Tue, Jul 23, 2013 at 11:29 AM, Jingang Wang [bitwjg@gmail.com](mailto:bitwjg@gmail.com) wrote: > > > > > Dear all, > > > > > > I found that after I conducted a delete by query operation, the matched > > > documents in the index have been deleted, while the size of the index > > > remains the same. > > > > > > Could anybody tell me why? Thanks in advance! > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/kg4EwfzZZzE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/kg4EwfzZZzE/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > Wang Jingang(ηŽ‹ι‡‘εˆš) > Ph.D. Candidate at > Lab of High Volume Language Information Processing & Cloud Computing > School of Computer Science > Beijing Institute of Technology > Beijing 100081 > P.R China > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/kg4EwfzZZzE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/kg4EwfzZZzE/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- Wang Jingang(ηŽ‹ι‡‘εˆš) Ph.D. Candidate at Lab of High Volume Language Information Processing & Cloud Computing School of Computer Science Beijing Institute of Technology Beijing 100081 P.R China -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:24am UTC](https://discuss.elastic.co/t/the-documents-deleted-while-the-size-of-the-index-kremains-the-same-after-using-delete-by-query/12893/6 "2017-07-06T02:24:52Z")