# ES index shows lots of deletes even though no delete operation **URL:** \ **Category:** Elasticsearch\ **Created:** [November 21, 2012, 11:19pm UTC](https://discuss.elastic.co/t/es-index-shows-lots-of-deletes-even-though-no-delete-operation/9789 "2012-11-21T23:19:21Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![T\_Vinod\_Gupta](https://avatars.discourse-cdn.com/v4/letter/t/fbc32d/32.png) [@T\_Vinod\_Gupta](https://discuss.elastic.co/u/T_Vinod_Gupta)\ **Post date:** [November 21, 2012, 11:19pm UTC](https://discuss.elastic.co/t/es-index-shows-lots-of-deletes-even-though-no-delete-operation/9789/1 "2012-11-21T23:19:21Z")
hi, i have a heavy index into which i am indexing documents.. lot of times, i am updating existing docs with new/updated field values.. when i run the stats api, i see that the index is accumulating bunch of deletes over time even though no explicit delete api is called. how is that possible? is there something happening under the covers? thanks -- ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [November 21, 2012, 11:24pm UTC](https://discuss.elastic.co/t/es-index-shows-lots-of-deletes-even-though-no-delete-operation/9789/2 "2012-11-21T23:24:37Z")
Updates in Lucene are essentially deletes + inserts. On Wed, Nov 21, 2012 at 3:19 PM, T Vinod Gupta [tvinod@readypulse.com](mailto:tvinod@readypulse.com)wrote: > hi, > i have a heavy index into which i am indexing documents.. lot of times, i > am updating existing docs with new/updated field values.. > > when i run the stats api, i see that the index is accumulating bunch of > deletes over time even though no explicit delete api is called. how is that > possible? is there something happening under the covers? > > thanks > > -- -- ---
**Author:** ![T\_Vinod\_Gupta](https://avatars.discourse-cdn.com/v4/letter/t/fbc32d/32.png) [@T\_Vinod\_Gupta](https://discuss.elastic.co/u/T_Vinod_Gupta)\ **Post date:** [November 21, 2012, 11:32pm UTC](https://discuss.elastic.co/t/es-index-shows-lots-of-deletes-even-though-no-delete-operation/9789/3 "2012-11-21T23:32:31Z")
i thought so too.. but then i created a test index with a test document and updated it.. running stats on test index didn't show any deletes. hence i got confused. assuming that updates causes deletes, does it make sense to have a daily/weekly cron job to run optimize with expunge deletes option to keep the index lightweight all the time? is that a recommended practice? if not, what are the recommendations in this regard? thanks On Wed, Nov 21, 2012 at 3:24 PM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > Updates in Lucene are essentially deletes + inserts. > > On Wed, Nov 21, 2012 at 3:19 PM, T Vinod Gupta [tvinod@readypulse.com](mailto:tvinod@readypulse.com)wrote: > > > hi, > > i have a heavy index into which i am indexing documents.. lot of times, i > > am updating existing docs with new/updated field values.. > > > > when i run the stats api, i see that the index is accumulating bunch of > > deletes over time even though no explicit delete api is called. how is that > > possible? is there something happening under the covers? > > > > thanks > > > > -- > > -- -- ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [November 21, 2012, 11:54pm UTC](https://discuss.elastic.co/t/es-index-shows-lots-of-deletes-even-though-no-delete-operation/9789/4 "2012-11-21T23:54:04Z")
The recommendation is to let Lucene's readers and segment do their own thing. Much work has been done to minimize the cost of a delete. If your search traffic has a consistent period of low activity, you can schedule an optimize for that time. -- Ivan On Wed, Nov 21, 2012 at 3:32 PM, T Vinod Gupta [tvinod@readypulse.com](mailto:tvinod@readypulse.com)wrote: > i thought so too.. but then i created a test index with a test document > and updated it.. running stats on test index didn't show any deletes. hence > i got confused. > assuming that updates causes deletes, does it make sense to have a > daily/weekly cron job to run optimize with expunge deletes option to keep > the index lightweight all the time? is that a recommended practice? if not, > what are the recommendations in this regard? > > thanks > > On Wed, Nov 21, 2012 at 3:24 PM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > > > Updates in Lucene are essentially deletes + inserts. > > > > On Wed, Nov 21, 2012 at 3:19 PM, T Vinod Gupta [tvinod@readypulse.com](mailto:tvinod@readypulse.com)wrote: > > > > > hi, > > > i have a heavy index into which i am indexing documents.. lot of times, > > > i am updating existing docs with new/updated field values.. > > > > > > when i run the stats api, i see that the index is accumulating bunch of > > > deletes over time even though no explicit delete api is called. how is that > > > possible? is there something happening under the covers? > > > > > > thanks > > > > > > -- > > > > -- > > -- -- ---
**Author:** ![Chris\_Male](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chris_male/32/2607_2.png) [@Chris\_Male](https://discuss.elastic.co/u/Chris_Male)\ **Post date:** [November 22, 2012, 3:58am UTC](https://discuss.elastic.co/t/es-index-shows-lots-of-deletes-even-though-no-delete-operation/9789/5 "2012-11-22T03:58:50Z")
I agree with Ivan, I'd let Lucene merge away the deleted documents when it's ready to. Optimizing isn't necessary and is very IO intensive. On Thursday, November 22, 2012 12:54:07 PM UTC+13, Ivan Brusic wrote: > The recommendation is to let Lucene's readers and segment do their own > thing. Much work has been done to minimize the cost of a delete. > > If your search traffic has a consistent period of low activity, you can > schedule an optimize for that time. > > -- > Ivan > > On Wed, Nov 21, 2012 at 3:32 PM, T Vinod Gupta \<[tvi...@readypulse.com](mailto:tvi...@readypulse.com)\ > > > wrote: > > > i thought so too.. but then i created a test index with a test document > > and updated it.. running stats on test index didn't show any deletes. hence > > i got confused. > > assuming that updates causes deletes, does it make sense to have a > > daily/weekly cron job to run optimize with expunge deletes option to keep > > the index lightweight all the time? is that a recommended practice? if not, > > what are the recommendations in this regard? > > > > thanks > > > > On Wed, Nov 21, 2012 at 3:24 PM, Ivan Brusic \<[iv...@brusic.com](mailto:iv...@brusic.com)\ > > > > > wrote: > > > > > Updates in Lucene are essentially deletes + inserts. > > > > > > On Wed, Nov 21, 2012 at 3:19 PM, T Vinod Gupta \<[tvi...@readypulse.com](mailto:tvi...@readypulse.com)\ > > > > > > > wrote: > > > > > > > hi, > > > > i have a heavy index into which i am indexing documents.. lot of times, > > > > i am updating existing docs with new/updated field values.. > > > > > > > > when i run the stats api, i see that the index is accumulating bunch of > > > > deletes over time even though no explicit delete api is called. how is that > > > > possible? is there something happening under the covers? > > > > > > > > thanks > > > > > > > > -- > > > > > > -- > > > > -- -- ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:03am UTC](https://discuss.elastic.co/t/es-index-shows-lots-of-deletes-even-though-no-delete-operation/9789/6 "2017-07-06T03:03:19Z")