# Frequently updating index entries **URL:** \ **Category:** Elasticsearch\ **Created:** [July 8, 2010, 1:42pm UTC](https://discuss.elastic.co/t/frequently-updating-index-entries/3082 "2010-07-08T13:42:00Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Michael\_Korbakov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/michael_korbakov/32/3199_2.png) [@Michael\_Korbakov](https://discuss.elastic.co/u/Michael_Korbakov)\ **Post date:** [July 8, 2010, 1:42pm UTC](https://discuss.elastic.co/t/frequently-updating-index-entries/3082/1 "2010-07-08T13:42:00Z")
Hi everyone. In our project we're going to have several millions of documents indexed by Elastic Search. Every day about 10% of all documents are updated. Fields updated are numerical values like view counts that we're going to use for scoring. I have some questions regarding this situation: 1. Is there any simpler way to update index entry other then fetching of whole document by \_source and then reindexing modified version with the same id? 2. How bad index fragmentation in this scenario could possibly hit me? 🙂 3. Any recommendations on index options for frequently updating fields? Thanks everyone! -- Michael Korbakov ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [July 8, 2010, 2:11pm UTC](https://discuss.elastic.co/t/frequently-updating-index-entries/3082/2 "2010-07-08T14:11:48Z")
On Thu, Jul 8, 2010 at 4:42 PM, Mykhailo Korbakov [rmihael@gmail.com](mailto:rmihael@gmail.com) wrote: > Hi everyone. > > In our project we're going to have several millions of documents > indexed by Elastic Search. Every day about 10% of all documents are > updated. Fields updated are numerical values like view counts that > we're going to use for scoring. > > I have some questions regarding this situation: > > 1. Is there any simpler way to update index entry other then fetching > of whole document by \_source and then reindexing modified version with > the same id? There is no way to do partial update, so you need to fetch, update and index back. > 1. How bad index fragmentation in this scenario could possibly hit me? 🙂 There will be fragmentation, but it will slowly be merged out. > 1. Any recommendations on index options for frequently updating fields? Nothing special for this case, its a very valid case. > Thanks everyone! > > -- Michael Korbakov ---
**Author:** ![Michael\_Korbakov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/michael_korbakov/32/3199_2.png) [@Michael\_Korbakov](https://discuss.elastic.co/u/Michael_Korbakov)\ **Post date:** [July 8, 2010, 2:26pm UTC](https://discuss.elastic.co/t/frequently-updating-index-entries/3082/3 "2010-07-08T14:26:22Z")
On Thu, Jul 8, 2010 at 5:11 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > On Thu, Jul 8, 2010 at 4:42 PM, Mykhailo Korbakov [rmihael@gmail.com](mailto:rmihael@gmail.com) wrote: > > > Hi everyone. > > > > In our project we're going to have several millions of documents > > indexed by Elastic Search. Every day about 10% of all documents are > > updated. Fields updated are numerical values like view counts that > > we're going to use for scoring. > > > > I have some questions regarding this situation: > > > > 1. Is there any simpler way to update index entry other then fetching > > of whole document by \_source and then reindexing modified version with > > the same id? > > There is no way to do partial update, so you need to fetch, update and index > back. > > > 1. How bad index fragmentation in this scenario could possibly hit me? 🙂 > > There will be fragmentation, but it will slowly be merged out. > > > 1. Any recommendations on index options for frequently updating fields? > > Nothing special for this case, its a very valid case. Thank you for answering, Shay. Just to make my soul completely calm down: is there any way to monitor fragmentation? May be I'll had to tune merger somehow to reduce it, etc. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [July 8, 2010, 2:30pm UTC](https://discuss.elastic.co/t/frequently-updating-index-entries/3082/4 "2010-07-08T14:30:54Z")
There isn't currently an API to return its value, but you can go to each shard storage, and check the number of files, they reflect the number of segments. There are parameters to control it (such as the merge\_factor), and there is an API to force "optimization". Post 0.9 I am going to provide a full set of API for index level "info" and "stats", in a similar manner current version provides for node. In them, this information will be exposed. -shay.banon On Thu, Jul 8, 2010 at 5:26 PM, Mykhailo Korbakov [rmihael@gmail.com](mailto:rmihael@gmail.com) wrote: > On Thu, Jul 8, 2010 at 5:11 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) > wrote: > > > On Thu, Jul 8, 2010 at 4:42 PM, Mykhailo Korbakov [rmihael@gmail.com](mailto:rmihael@gmail.com) > > wrote: > > > > > Hi everyone. > > > > > > In our project we're going to have several millions of documents > > > indexed by Elastic Search. Every day about 10% of all documents are > > > updated. Fields updated are numerical values like view counts that > > > we're going to use for scoring. > > > > > > I have some questions regarding this situation: > > > > > > 1. Is there any simpler way to update index entry other then fetching > > > of whole document by \_source and then reindexing modified version with > > > the same id? > > > > There is no way to do partial update, so you need to fetch, update and > > index > > back. > > > > > 1. How bad index fragmentation in this scenario could possibly hit me? > > > 🙂 > > > > There will be fragmentation, but it will slowly be merged out. > > > > > 1. Any recommendations on index options for frequently updating fields? > > > > Nothing special for this case, its a very valid case. > > Thank you for answering, Shay. > > Just to make my soul completely calm down: is there any way to monitor > fragmentation? May be I'll had to tune merger somehow to reduce it, > etc. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:22am UTC](https://discuss.elastic.co/t/frequently-updating-index-entries/3082/5 "2017-07-06T04:22:29Z")