Most efficient way to bulk update index?


Here is my requirement.
This is what I need to do ...
if (indexedField1.value contains pattern1) indexedField1.value = newValue; add a new indexedField2
I do realize there is a programmatic way of doing this in Java for example:
1. Fetch only indexedField1 using _source filtering
2. Check the value
3. Create _bulk update and create statements
4. Execute the _bulk call

I wanted to verify if the above would work.
More importantly, is there a better way to do this, for example, entire within elastic search, hence more efficient?


Any thoughts on this will be very helpful.

Using _source filtering does not makes it fast. Actually, it adds overhead since Elasticsearch needs to parse the _source, apply the filtering logic and serialize a new document.

The most efficient way of doing what you are describing is by either using the Reindex API with a script or using the Update By Query API also with a script.

Both of which could be leveraged if you can translate indexedField1.value contains pattern1 into a query.

Thank you, very useful.
Yea I figured anything that can be natively done within ES would be preferred. I just wasnt sure of it. More importantly Im not too familiar with the concept of scripts.
I guess here is my opportunity to explore, learn and apply it.

Once again, appreciate your response.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.