Preserving Inner Objects During Re-Index of Parent Document

Nariman_Haghighi · June 22, 2013, 3:22pm

We have a JSON document (Media) that has two inner objects (GeoCodeResult
and LocalTime).

We first index the Media document and then rely on secondary background
processes to populate the two inner objects separately.

We need the three to be combined into a single ES type for performance
reasons.

The Media documents are re-indexed from their original source from time to
time, and when this happens the two inner objects that were previously
populated are reset back to null.

Is it possible to update the Media documents while preserving the two inner
objects?

I'm picturing something like an update script that can be provided to the
index operation that preserves the two inner objects unless non-null values
are found in the new parent document.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Nariman_Haghighi · June 23, 2013, 2:50pm

Seems like this is now supported via bulk partial updates using the upsert
scripts:

Partial updates and nested types:
http://elasticsearch-users.115913.n3.nabble.com/partial-update-and-nested-type-td3959065.html

Added support for the update operation in the bulk api:

github.com/elastic/elasticsearch

Added support for the update operation in the bulk api.

committed 02:03PM - 10 May 13 UTC

martijnvg

+1181 -313

Update requests can now be put in the bulk api. All update request options are s…upported. Example usage: ``` curl -XPOST 'localhost:9200/_bulk' --date-binary @bulk.json ``` Contents of bulk.json that contains two update request items: ``` { "update" : {"_id" : "1", "_type" : "type1", "_index" : "index1", "_retry_on_conflict" : 3} } { "doc" : {"field" : "value"} } { "update" : { "_id" : "0", "_type" : "type1", "_index" : "index1", "_retry_on_conflict" : 3} } { "script" : "counter += param1", "lang" : "js", "params" : {"param1" : 1}, "upsert" : {"counter" : 1}} ``` The `doc`, `upsert` and all script related options are part of the payload. The `retry_on_conflict` option is part of the header. Closes #2982

Is this correct? How does this approach perform in comparison to the
standard index API call?

On Saturday, June 22, 2013 11:22:45 AM UTC-4, Nariman Haghighi wrote:

We have a JSON document (Media) that has two inner objects (GeoCodeResult
and LocalTime).

We first index the Media document and then rely on secondary background
processes to populate the two inner objects separately.

We need the three to be combined into a single ES type for performance
reasons.

The Media documents are re-indexed from their original source from time to
time, and when this happens the two inner objects that were previously
populated are reset back to null.

Is it possible to update the Media documents while preserving the two
inner objects?

I'm picturing something like an update script that can be provided to the
index operation that preserves the two inner objects unless non-null values
are found in the new parent document.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.