I need to update all documents in my elastic index by adding new field for every doc. I need to get all docs from index by fastes way. My update method get all docs -> save to Java Map.class -> get every doc from Map -> get from it one field and his value -> add by this join field new field with value and send some doc chunk to elastic by bulk. And slowest stage of my update operation it's getting all docs from index. I'm trying to use parallel concurrent threads where every thread get data docs using (from, size) in SearchSourceBuilder, but I still run into performance ceiling. I'm try using scroll api and this slowly then parallel thread. May be exist way to use stream what can read every doc fast and without store docs in memory or another fast way for get all docs from elastic index? What another way to get large amount of data from elastic index?
Do you need to do this in a custom program? If you don't need any other external system (RDBMS, etc) and can do the work in a painless script, using update by query might be quicker.
Yes, I need update doc in my java method. May be I need use data streams?
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.