How does partial update work with custom _source?

Elastic Stack Elasticsearch
1

I am seeing a strange behavior with partial update in ES 0.19.2.
I have an index where _source has only few selected fields whereas I am
indexing many more fields.
The problem happens when I partially update documents. After the partial
update, I noticed that a
bunch of docs looses indexed fields which are not in _source.

I am not sure but the source code herehttps://github.com/medcl/elasticsearch-partialupdate/blob/master/src/main/java/org/elasticsearch/plugin/infinitbyte/PartialUpdateRestAction.java seems
to show that _source is taken as the document on which partial update is
applied
and later the resulting object indexed. If _source doesn't have all indexed
fields, then this logic would be buggy.
Can somebody shed more light on it.

Thanks a lot,
Vinay

2

Anybody?

On Wednesday, April 25, 2012 6:13:29 PM UTC-7, revdev wrote:

I am seeing a strange behavior with partial update in ES 0.19.2.
I have an index where _source has only few selected fields whereas I am
indexing many more fields.
The problem happens when I partially update documents. After the partial
update, I noticed that a
bunch of docs looses indexed fields which are not in _source.

I am not sure but the source code herehttps://github.com/medcl/elasticsearch-partialupdate/blob/master/src/main/java/org/elasticsearch/plugin/infinitbyte/PartialUpdateRestAction.java seems
to show that _source is taken as the document on which partial update is
applied
and later the resulting object indexed. If _source doesn't have all
indexed fields, then this logic would be buggy.
Can somebody shed more light on it.

Thanks a lot,
Vinay

3

Hi,

If I understood you correctly, you simply need to make sure your _source
has all original fields.

Otis

On Wednesday, April 25, 2012 9:13:29 PM UTC-4, revdev wrote:

I am seeing a strange behavior with partial update in ES 0.19.2.
I have an index where _source has only few selected fields whereas I am
indexing many more fields.
The problem happens when I partially update documents. After the partial
update, I noticed that a
bunch of docs looses indexed fields which are not in _source.

I am not sure but the source code herehttps://github.com/medcl/elasticsearch-partialupdate/blob/master/src/main/java/org/elasticsearch/plugin/infinitbyte/PartialUpdateRestAction.java seems
to show that _source is taken as the document on which partial update is
applied
and later the resulting object indexed. If _source doesn't have all
indexed fields, then this logic would be buggy.
Can somebody shed more light on it.

Thanks a lot,
Vinay

4

Yes doing that I solved the problem but the problem is that I had to double
the capacity to support _source with all fields. Apart from that, also
reindex all the data.
I am wondering if there is any other way to implement partialUpdate.
If not, can this be please mentioned in the ES docs so that others don't
repeat the same mistake.

On Thursday, April 26, 2012 1:37:58 PM UTC-7, Otis Gospodnetic wrote:

Hi,

If I understood you correctly, you simply need to make sure your _source
has all original fields.

Otis

On Wednesday, April 25, 2012 9:13:29 PM UTC-4, revdev wrote:

I am seeing a strange behavior with partial update in ES 0.19.2.
I have an index where _source has only few selected fields whereas I am
indexing many more fields.
The problem happens when I partially update documents. After the partial
update, I noticed that a
bunch of docs looses indexed fields which are not in _source.

I am not sure but the source code herehttps://github.com/medcl/elasticsearch-partialupdate/blob/master/src/main/java/org/elasticsearch/plugin/infinitbyte/PartialUpdateRestAction.java seems
to show that _source is taken as the document on which partial update is
applied
and later the resulting object indexed. If _source doesn't have all
indexed fields, then this logic would be buggy.
Can somebody shed more light on it.

Thanks a lot,
Vinay

5

I think its mentioned in the update API, where its stated that the document
is read and then reindexed (several times actually on that page).

On Thu, Apr 26, 2012 at 11:58 PM, revdev clickingcam@gmail.com wrote:

Yes doing that I solved the problem but the problem is that I had to
double the capacity to support _source with all fields. Apart from that,
also reindex all the data.
I am wondering if there is any other way to implement partialUpdate.
If not, can this be please mentioned in the ES docs so that others don't
repeat the same mistake.

On Thursday, April 26, 2012 1:37:58 PM UTC-7, Otis Gospodnetic wrote:

Hi,

If I understood you correctly, you simply need to make sure your _source
has all original fields.

Otis

On Wednesday, April 25, 2012 9:13:29 PM UTC-4, revdev wrote:

I am seeing a strange behavior with partial update in ES 0.19.2.
I have an index where _source has only few selected fields whereas I am
indexing many more fields.
The problem happens when I partially update documents. After the partial
update, I noticed that a
bunch of docs looses indexed fields which are not in _source.

I am not sure but the source code herehttps://github.com/medcl/elasticsearch-partialupdate/blob/master/src/main/java/org/elasticsearch/plugin/infinitbyte/PartialUpdateRestAction.java seems
to show that _source is taken as the document on which partial update is
applied
and later the resulting object indexed. If _source doesn't have all
indexed fields, then this logic would be buggy.
Can somebody shed more light on it.

Thanks a lot,
Vinay

6

I am sorry, I couldn't find it in the docs. Can you please check it again.
If you could mention the pre-condition that all fields are required to be
present in _source for partial updates to work, it would make much more
sense.
Thanks!

On Sunday, April 29, 2012 9:32:03 AM UTC-7, kimchy wrote:

I think its mentioned in the update API, where its stated that the
document is read and then reindexed (several times actually on that page).

On Thu, Apr 26, 2012 at 11:58 PM, revdev wrote:

Yes doing that I solved the problem but the problem is that I had to
double the capacity to support _source with all fields. Apart from that,
also reindex all the data.
I am wondering if there is any other way to implement partialUpdate.
If not, can this be please mentioned in the ES docs so that others don't
repeat the same mistake.

On Thursday, April 26, 2012 1:37:58 PM UTC-7, Otis Gospodnetic wrote:

Hi,

If I understood you correctly, you simply need to make sure your _source
has all original fields.

Otis

On Wednesday, April 25, 2012 9:13:29 PM UTC-4, revdev wrote:

I am seeing a strange behavior with partial update in ES 0.19.2.
I have an index where _source has only few selected fields whereas I am
indexing many more fields.
The problem happens when I partially update documents. After the
partial update, I noticed that a
bunch of docs looses indexed fields which are not in _source.

I am not sure but the source code herehttps://github.com/medcl/elasticsearch-partialupdate/blob/master/src/main/java/org/elasticsearch/plugin/infinitbyte/PartialUpdateRestAction.java seems
to show that _source is taken as the document on which partial update is
applied
and later the resulting object indexed. If _source doesn't have all
indexed fields, then this logic would be buggy.
Can somebody shed more light on it.

Thanks a lot,
Vinay