How to update multiple documents in ElasticSearch with conditions?

Elastic Stack Elasticsearch
1

Hello, I'm a newbie in ElasticSearch,
I have a problems, I want tagging documents stored in ElasticSearch with specific conditions,
Ex: I want to add new attributes tags = "hight" for any document have height attribute >= 50.
But I read on docs, i have'nt any solution ?
Please help me? Thanks all. Sorry about my bad english.

2

Hi @ndhcoder,

I think what you are after is the update_by_query API, which allows updating existing docs based on a search specification.

2 Likes
3

Thanks you.
I have one more question ? With very large data set (milions or bilions documents, the update_by_query API can be use ? how about performance ?
Example: user will add condition dynamic to tagging all documents with their documents logs very large.

Thanks you so much !!!

4

Hi @ndhcoder,

it can be used for large data sets too (it uses a scroll query to efficiently run through the dataset), but there are a couple of things to be aware of:

  1. It is not resilient. This means that if either the node where update by query runs or one of the nodes involved in the query dies or are restarted while this runs, the process will die with it.
  2. The scroll query will hold back the consistent view of data that it was started against. If an update by query touches all docs, this may end up taking double disk space or more until the query part is finished.

I think that in your case you might be able to add a query clause to only update docs that do not already have the tag. This would allow you to rerun the update by query in case anything goes wrong.

It could also allow you to add max_docs to run the update in smaller batches, freeing up disk space sooner instead, if you desire.

Performance-wise, it depends somewhat on the size of the docs being modified, since any update replaces the entire old document with the new document. I would advise to try it out to get a feel for the run time. There are numerous options for tuning this if performance is not adequate.

2 Likes
5

Thanks you

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.