# How does partial update work with custom \_source? **URL:** \ **Category:** Elasticsearch\ **Created:** [April 26, 2012, 1:13am UTC](https://discuss.elastic.co/t/how-does-partial-update-work-with-custom--source/7466 "2012-04-26T01:13:29Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![revdev](https://avatars.discourse-cdn.com/v4/letter/r/ecb155/32.png) [@revdev](https://discuss.elastic.co/u/revdev)\ **Post date:** [April 26, 2012, 1:13am UTC](https://discuss.elastic.co/t/how-does-partial-update-work-with-custom--source/7466/1 "2012-04-26T01:13:29Z")
I am seeing a strange behavior with partial update in ES 0.19.2. I have an index where \_source has only few selected fields whereas I am indexing many more fields. The problem happens when I partially update documents. After the partial update, I noticed that a bunch of docs looses indexed fields which are not in \_source. I am not sure but the source code here[https://github.com/medcl/elasticsearch-partialupdate/blob/master/src/main/java/org/elasticsearch/plugin/infinitbyte/PartialUpdateRestAction.java](https://github.com/medcl/elasticsearch-partialupdate/blob/master/src/main/java/org/elasticsearch/plugin/infinitbyte/PartialUpdateRestAction.java) seems to show that \_source is taken as the document on which partial update is applied and later the resulting object indexed. If \_source doesn't have all indexed fields, then this logic would be buggy. Can somebody shed more light on it. Thanks a lot, Vinay ---
**Author:** ![revdev](https://avatars.discourse-cdn.com/v4/letter/r/ecb155/32.png) [@revdev](https://discuss.elastic.co/u/revdev)\ **Post date:** [April 26, 2012, 5:32pm UTC](https://discuss.elastic.co/t/how-does-partial-update-work-with-custom--source/7466/2 "2012-04-26T17:32:32Z")
Anybody? On Wednesday, April 25, 2012 6:13:29 PM UTC-7, revdev wrote: > I am seeing a strange behavior with partial update in ES 0.19.2. > I have an index where \_source has only few selected fields whereas I am > indexing many more fields. > The problem happens when I partially update documents. After the partial > update, I noticed that a > bunch of docs looses indexed fields which are not in \_source. > > I am not sure but the source code here[https://github.com/medcl/elasticsearch-partialupdate/blob/master/src/main/java/org/elasticsearch/plugin/infinitbyte/PartialUpdateRestAction.java](https://github.com/medcl/elasticsearch-partialupdate/blob/master/src/main/java/org/elasticsearch/plugin/infinitbyte/PartialUpdateRestAction.java) seems > to show that \_source is taken as the document on which partial update is > applied > and later the resulting object indexed. If \_source doesn't have all > indexed fields, then this logic would be buggy. > Can somebody shed more light on it. > > Thanks a lot, > Vinay ---
**Author:** ![otisg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/otisg/32/492_2.png) [@otisg](https://discuss.elastic.co/u/otisg)\ **Post date:** [April 26, 2012, 8:37pm UTC](https://discuss.elastic.co/t/how-does-partial-update-work-with-custom--source/7466/3 "2012-04-26T20:37:58Z")
Hi, If I understood you correctly, you simply need to make sure your \_source has _all_ original fields. Otis On Wednesday, April 25, 2012 9:13:29 PM UTC-4, revdev wrote: > I am seeing a strange behavior with partial update in ES 0.19.2. > I have an index where \_source has only few selected fields whereas I am > indexing many more fields. > The problem happens when I partially update documents. After the partial > update, I noticed that a > bunch of docs looses indexed fields which are not in \_source. > > I am not sure but the source code here[https://github.com/medcl/elasticsearch-partialupdate/blob/master/src/main/java/org/elasticsearch/plugin/infinitbyte/PartialUpdateRestAction.java](https://github.com/medcl/elasticsearch-partialupdate/blob/master/src/main/java/org/elasticsearch/plugin/infinitbyte/PartialUpdateRestAction.java) seems > to show that \_source is taken as the document on which partial update is > applied > and later the resulting object indexed. If \_source doesn't have all > indexed fields, then this logic would be buggy. > Can somebody shed more light on it. > > Thanks a lot, > Vinay ---
**Author:** ![revdev](https://avatars.discourse-cdn.com/v4/letter/r/ecb155/32.png) [@revdev](https://discuss.elastic.co/u/revdev)\ **Post date:** [April 26, 2012, 8:58pm UTC](https://discuss.elastic.co/t/how-does-partial-update-work-with-custom--source/7466/4 "2012-04-26T20:58:21Z")
Yes doing that I solved the problem but the problem is that I had to double the capacity to support \_source with all fields. Apart from that, also reindex all the data. I am wondering if there is any other way to implement partialUpdate. If not, can this be please mentioned in the ES docs so that others don't repeat the same mistake. On Thursday, April 26, 2012 1:37:58 PM UTC-7, Otis Gospodnetic wrote: > Hi, > > If I understood you correctly, you simply need to make sure your \_source > has _all_ original fields. > > Otis > > On Wednesday, April 25, 2012 9:13:29 PM UTC-4, revdev wrote: > > > I am seeing a strange behavior with partial update in ES 0.19.2. > > I have an index where \_source has only few selected fields whereas I am > > indexing many more fields. > > The problem happens when I partially update documents. After the partial > > update, I noticed that a > > bunch of docs looses indexed fields which are not in \_source. > > > > I am not sure but the source code here[https://github.com/medcl/elasticsearch-partialupdate/blob/master/src/main/java/org/elasticsearch/plugin/infinitbyte/PartialUpdateRestAction.java](https://github.com/medcl/elasticsearch-partialupdate/blob/master/src/main/java/org/elasticsearch/plugin/infinitbyte/PartialUpdateRestAction.java) seems > > to show that \_source is taken as the document on which partial update is > > applied > > and later the resulting object indexed. If \_source doesn't have all > > indexed fields, then this logic would be buggy. > > Can somebody shed more light on it. > > > > Thanks a lot, > > Vinay ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [April 29, 2012, 4:32pm UTC](https://discuss.elastic.co/t/how-does-partial-update-work-with-custom--source/7466/5 "2012-04-29T16:32:03Z")
I think its mentioned in the update API, where its stated that the document is read and then reindexed (several times actually on that page). On Thu, Apr 26, 2012 at 11:58 PM, revdev [clickingcam@gmail.com](mailto:clickingcam@gmail.com) wrote: > Yes doing that I solved the problem but the problem is that I had to > double the capacity to support \_source with all fields. Apart from that, > also reindex all the data. > I am wondering if there is any other way to implement partialUpdate. > If not, can this be please mentioned in the ES docs so that others don't > repeat the same mistake. > > On Thursday, April 26, 2012 1:37:58 PM UTC-7, Otis Gospodnetic wrote: > > > Hi, > > > > If I understood you correctly, you simply need to make sure your \_source > > has _all_ original fields. > > > > Otis > > > > On Wednesday, April 25, 2012 9:13:29 PM UTC-4, revdev wrote: > > > > > I am seeing a strange behavior with partial update in ES 0.19.2. > > > I have an index where \_source has only few selected fields whereas I am > > > indexing many more fields. > > > The problem happens when I partially update documents. After the partial > > > update, I noticed that a > > > bunch of docs looses indexed fields which are not in \_source. > > > > > > I am not sure but the source code here[https://github.com/medcl/elasticsearch-partialupdate/blob/master/src/main/java/org/elasticsearch/plugin/infinitbyte/PartialUpdateRestAction.java](https://github.com/medcl/elasticsearch-partialupdate/blob/master/src/main/java/org/elasticsearch/plugin/infinitbyte/PartialUpdateRestAction.java) seems > > > to show that \_source is taken as the document on which partial update is > > > applied > > > and later the resulting object indexed. If \_source doesn't have all > > > indexed fields, then this logic would be buggy. > > > Can somebody shed more light on it. > > > > > > Thanks a lot, > > > Vinay ---
**Author:** ![revdev](https://avatars.discourse-cdn.com/v4/letter/r/ecb155/32.png) [@revdev](https://discuss.elastic.co/u/revdev)\ **Post date:** [April 29, 2012, 10:45pm UTC](https://discuss.elastic.co/t/how-does-partial-update-work-with-custom--source/7466/6 "2012-04-29T22:45:16Z")
I am sorry, I couldn't find it in the docs. Can you please check it again. If you could mention the pre-condition that all fields are required to be present in \_source for partial updates to work, it would make much more sense. Thanks! On Sunday, April 29, 2012 9:32:03 AM UTC-7, kimchy wrote: > I think its mentioned in the update API, where its stated that the > document is read and then reindexed (several times actually on that page). > > On Thu, Apr 26, 2012 at 11:58 PM, revdev wrote: > > > Yes doing that I solved the problem but the problem is that I had to > > double the capacity to support \_source with all fields. Apart from that, > > also reindex all the data. > > I am wondering if there is any other way to implement partialUpdate. > > If not, can this be please mentioned in the ES docs so that others don't > > repeat the same mistake. > > > > On Thursday, April 26, 2012 1:37:58 PM UTC-7, Otis Gospodnetic wrote: > > > > > Hi, > > > > > > If I understood you correctly, you simply need to make sure your \_source > > > has _all_ original fields. > > > > > > Otis > > > > > > On Wednesday, April 25, 2012 9:13:29 PM UTC-4, revdev wrote: > > > > > > > I am seeing a strange behavior with partial update in ES 0.19.2. > > > > I have an index where \_source has only few selected fields whereas I am > > > > indexing many more fields. > > > > The problem happens when I partially update documents. After the > > > > partial update, I noticed that a > > > > bunch of docs looses indexed fields which are not in \_source. > > > > > > > > I am not sure but the source code here[https://github.com/medcl/elasticsearch-partialupdate/blob/master/src/main/java/org/elasticsearch/plugin/infinitbyte/PartialUpdateRestAction.java](https://github.com/medcl/elasticsearch-partialupdate/blob/master/src/main/java/org/elasticsearch/plugin/infinitbyte/PartialUpdateRestAction.java) seems > > > > to show that \_source is taken as the document on which partial update is > > > > applied > > > > and later the resulting object indexed. If \_source doesn't have all > > > > indexed fields, then this logic would be buggy. > > > > Can somebody shed more light on it. > > > > > > > > Thanks a lot, > > > > Vinay ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:30am UTC](https://discuss.elastic.co/t/how-does-partial-update-work-with-custom--source/7466/7 "2017-07-06T03:30:35Z")