# Conditional update to the document **URL:** \ **Category:** Elasticsearch\ **Created:** [November 4, 2016, 2:06am UTC](https://discuss.elastic.co/t/conditional-update-to-the-document/64964 "2016-11-04T02:06:45Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![animageofmine](https://avatars.discourse-cdn.com/v4/letter/a/7feea3/32.png) [@animageofmine](https://discuss.elastic.co/u/animageofmine)\ **Post date:** [November 4, 2016, 2:06am UTC](https://discuss.elastic.co/t/conditional-update-to-the-document/64964/1 "2016-11-04T02:06:45Z")
Hello, If I wanted to update an existing document completely (not partially), if it satisfies the condition. See an example below: Index: Twitter Type: tweet 1. Add tweet and lastupdated fields to document 1 as below. Please note that the lastupdated field is of type date and has epoch value. 2. I want to update the document only if the epoch time is greater than the current existing time (which is 1478218039000). I tried using the script, but I get an error that doc and script cannot be used together (see error below). I also tried upsert with a script, but no luck. See a couple of queries that I tried, but in vain. Can someone please share if this is possible in ElasticSearch? Option 1 with script upsert that didn't work. ES returns success, but does not update the doc. It looks like it only either executes script or upsert. ``` POST /twitter/tweet/1/_update { "scripted_upsert":true, "script": { "lang": "painless", "inline" : "if (ctx._source.lastupdated > params.currdate) { ctx.op = 'noop'}", "params": { "currdate": 1478219029000 } }, "upsert" : { "tweet" : "tweet2", "lastupdated" : 1478219029000 } } ``` Option 2 with script and doc that didn't work as well ``` POST /twitter/tweet/1/_update { "script": { "lang": "painless", "inline" : "if (ctx._source.lastupdated > params.currdate) { ctx.op = 'noop'}", "params": { "currdate": 1478219029000 } }, "doc" : { "tweet" : "tweet2", "lastupdated" : 1478219029000 } } ``` Error: ``` { "error": { "root_cause": [ { "type": "action_request_validation_exception", "reason": "Validation Failed: 1: can't provide both script and doc;" } ], "type": "action_request_validation_exception", "reason": "Validation Failed: 1: can't provide both script and doc;" }, "status": 400 } ``` A lot of other folks seem to have requested this information, but I can't seem to find an answer. See [github](https://github.com/elastic/elasticsearch/issues/16439) and [stackoverflow](http://stackoverflow.com/questions/31599068/using-a-script-to-conditionally-update-a-document-in-elasticsearch) for couple of requests. Thank you for your help in advance. ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [November 4, 2016, 11:04am UTC](https://discuss.elastic.co/t/conditional-update-to-the-document/64964/2 "2016-11-04T11:04:56Z")
The execution choice is it either applies the doc or relies on a script. I think you're assuming you can use a bit of both (the script to decide if noop is set and the doc to patch the new data). If you opt for scripted\_upsert:true your script has to do all the work e.g. ``` POST /twitter/tweet/1/_update { "scripted_upsert":true, "script": { "lang": "painless", "inline" : "if (ctx._source.lastupdated > params.currdate) { ctx.op = 'noop'} ctx._source.tweet=params.tweet", "params": { "currdate": 1478219029000, "tweet":"tweet2" } } } ``` ---
**Author:** ![animageofmine](https://avatars.discourse-cdn.com/v4/letter/a/7feea3/32.png) [@animageofmine](https://discuss.elastic.co/u/animageofmine)\ **Post date:** [November 4, 2016, 3:08pm UTC](https://discuss.elastic.co/t/conditional-update-to-the-document/64964/3 "2016-11-04T15:08:16Z")
Mark, Thank you for the clarification and the example. I should have mentioned that I tried this earlier. While this would work, the problem with this approach is that the script gets uglier as the number of fields grows (think if 100s). Our use case is that we want to ingest data concurrently and sometimes it is possible that the same document can be updated multiple times and can arrive out of order to elasticsearch. We want to update the document only if it is newer by checking the timestamp like it is shown in the example above. Also, I was planning to do this via Java APIs. 1. Is there a cleaner way of doing this? (thinking of Java/Scala APIs) 2. Under the hood, is this an atomic operation? Are there two round trip calls or just one? ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [November 4, 2016, 3:27pm UTC](https://discuss.elastic.co/t/conditional-update-to-the-document/64964/4 "2016-11-04T15:27:18Z")
I hear you on the complex scripts. Perhaps you could instead rely on the elasticsearch version checking? Ordinarily it increments the version numbers for you but perhaps you could supply them instead as timestamps? ---
**Author:** ![animageofmine](https://avatars.discourse-cdn.com/v4/letter/a/7feea3/32.png) [@animageofmine](https://discuss.elastic.co/u/animageofmine)\ **Post date:** [November 4, 2016, 3:48pm UTC](https://discuss.elastic.co/t/conditional-update-to-the-document/64964/5 "2016-11-04T15:48:07Z")
I am guessing you meant the version number ("\_version": ). 1. Is there a way to supply our own version numbers? 2. Can you check existing version number and update the document if the incoming document's timestamp/version is greater than existing one in one call? In other words, can it lead to race condition in case of concurrent requests? An example would be very useful here. Thank you. ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [November 4, 2016, 3:50pm UTC](https://discuss.elastic.co/t/conditional-update-to-the-document/64964/6 "2016-11-04T15:50:53Z")
I've not used the feature myself but here's a good place to start: [https://www.elastic.co/blog/elasticsearch-versioning-support](https://www.elastic.co/blog/elasticsearch-versioning-support) ---
**Author:** ![Dan\_Markhasin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dan_markhasin/32/14187_2.png) [@Dan\_Markhasin](https://discuss.elastic.co/u/Dan_Markhasin)\ **Post date:** [December 28, 2016, 2:42pm UTC](https://discuss.elastic.co/t/conditional-update-to-the-document/64964/7 "2016-12-28T14:42:35Z")
Hi, Have you found a solution for this scenario? We are facing the same challenge of performing conditional updates on documents (also based on a timestamp or a date field), and it doesn't look like there is any viable solution for this at the moment. ---
**Author:** ![animageofmine](https://avatars.discourse-cdn.com/v4/letter/a/7feea3/32.png) [@animageofmine](https://discuss.elastic.co/u/animageofmine)\ **Post date:** [December 28, 2016, 5:20pm UTC](https://discuss.elastic.co/t/conditional-update-to-the-document/64964/8 "2016-12-28T17:20:14Z")
Hi Dan, Try external versioning. Basically, you use timestamp (epoch) as your version for the document. If the timestamp is lower than existing document, the request will be rejected. > **[Elasticsearch Versioning Support | Elastic](https://www.elastic.co/blog/elasticsearch-versioning-support)** > > Elasticsearch Versioning Support One of the key principles behind Elasticsearch is to allow you to make the most out of your data. Historically, search was a read-only enterprise where a search engine... I might be able to post an example later on, if you still need help. ---
**Author:** ![Dan\_Markhasin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dan_markhasin/32/14187_2.png) [@Dan\_Markhasin](https://discuss.elastic.co/u/Dan_Markhasin)\ **Post date:** [December 28, 2016, 5:28pm UTC](https://discuss.elastic.co/t/conditional-update-to-the-document/64964/9 "2016-12-28T17:28:54Z")
Thanks, I have looked into it but the update API does not support external versioning (it is explicitly mentioned in the documentation). So unless I am missing something, it looks like an obvious feature such as "update the document if " is simply missing from ElasticSearch. ---
**Author:** ![animageofmine](https://avatars.discourse-cdn.com/v4/letter/a/7feea3/32.png) [@animageofmine](https://discuss.elastic.co/u/animageofmine)\ **Post date:** [December 28, 2016, 8:48pm UTC](https://discuss.elastic.co/t/conditional-update-to-the-document/64964/10 "2016-12-28T20:48:45Z")
Do you need to use update API? Or can you simply consider it as insert with new version? ---
**Author:** ![Dan\_Markhasin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dan_markhasin/32/14187_2.png) [@Dan\_Markhasin](https://discuss.elastic.co/u/Dan_Markhasin)\ **Post date:** [December 29, 2016, 8:07am UTC](https://discuss.elastic.co/t/conditional-update-to-the-document/64964/11 "2016-12-29T08:07:16Z")
I do, because I am dealing with partial updates. Consider the following scenario, where I have an indexed document that looks like this: { "\_id":9999999 "name":"John", "Address":"Some Address", "Phone":12345568, "UpdateDate":2016-01-01T10:00:00.000 } And when John's phone number is updated, I will get an incremental update that looks like this (let's assume \_id is John's social security number, and it is unique): { "\_id":9999999 "Phone":12341234 "UpdateDate":2016-09-01T12:00:00.000 } So, using the update API I simply update the UpdateDate and Phone of John's record, and that's it. But what if I get a record with an older UpdateDate than what I already have? { "\_id":9999999 "Phone":12344321 "UpdateDate":2015-03-06T13:00:00.000 } In this case I don't want to do anything, since this is obviously an out of date record, which I want to discard. I was really hoping it would be possible to use a script like "if (ctx.\_source.UpdateDate \> UpdateDate) { ctx.op = 'none'} - otherwise use the new partial doc. But if a document already exists, Elastic will ignore the attached document and always run the script - which means I need to encapsulate the entire update logic inside the script, which means the script itself has to be programatically generated based on the document I am trying to update, i.e.: if (ctx.\_source.UpdateDate \> '2015-03-06T13:00:00.000') { ctx.op = 'none' } else { ctx.\_source.Phone = 12341234} ---
**Author:** ![animageofmine](https://avatars.discourse-cdn.com/v4/letter/a/7feea3/32.png) [@animageofmine](https://discuss.elastic.co/u/animageofmine)\ **Post date:** [December 29, 2016, 5:26pm UTC](https://discuss.elastic.co/t/conditional-update-to-the-document/64964/12 "2016-12-29T17:26:00Z")
This is how I started my journey with updates. I think one of the best options I know of is to perform a complete update with new data using external versioning instead of partial update. This way, it is supported by ES and you don't have to deal with additional logic / locking in your code. ---
**Author:** ![Bgolla](https://avatars.discourse-cdn.com/v4/letter/b/b3f665/32.png) [@Bgolla](https://discuss.elastic.co/u/Bgolla)\ **Post date:** [May 25, 2017, 8:27pm UTC](https://discuss.elastic.co/t/conditional-update-to-the-document/64964/13 "2017-05-25T20:27:01Z")
Hi, Were you able to solve the issue? i have a similar problem and I am looking for solution for conditional upserts. Thanks! ---
**Author:** ![Dan\_Markhasin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dan_markhasin/32/14187_2.png) [@Dan\_Markhasin](https://discuss.elastic.co/u/Dan_Markhasin)\ **Post date:** [May 26, 2017, 6:58am UTC](https://discuss.elastic.co/t/conditional-update-to-the-document/64964/14 "2017-05-26T06:58:08Z")
The only working solution so far is to auto-generate the update script.. ---
**Author:** ![Bgolla](https://avatars.discourse-cdn.com/v4/letter/b/b3f665/32.png) [@Bgolla](https://discuss.elastic.co/u/Bgolla)\ **Post date:** [May 26, 2017, 3:36pm UTC](https://discuss.elastic.co/t/conditional-update-to-the-document/64964/15 "2017-05-26T15:36:23Z")
> [@Dan\_Markhasin](#): > > The only working solution so far is to auto-generate the update script.. Thanks for the reply Dan. If possible can you please send me the syntax for update scripts example. ---
**Author:** ![Dan\_Markhasin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dan_markhasin/32/14187_2.png) [@Dan\_Markhasin](https://discuss.elastic.co/u/Dan_Markhasin)\ **Post date:** [June 3, 2017, 4:09pm UTC](https://discuss.elastic.co/t/conditional-update-to-the-document/64964/16 "2017-06-03T16:09:05Z")
It's something along the lines of the following: ``` filter { ruby { code => 'field_name = "update_script" condCol = event.get("condCol") arr = [] script = "if (ctx._source.#{condCol} > params.event.get(\"#{condCol}\")) { ctx.op = \"none\" } else {}" event.to_hash.each do |key,value| next if key.start_with?("@") arr.push("ctx._source.#{key} = params.event.get(\"#{key}\")") end updates = arr.join(";") script.sub!("",updates) event.set(field_name,script)' } } ``` And then in the elasticsearch output plugin: `script => "%{update_script}"` This assumes there is a "condCol" field in the input that describes which field should be used for the condition. Also, this assumes the data is flat (or that you don't care about nesting) - there is no deep merge going on here, top level values get fully overwritten. ---
**Author:** ![Stefan\_Rogin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stefan_rogin/32/18853_2.png) [@Stefan\_Rogin](https://discuss.elastic.co/u/Stefan_Rogin)\ **Post date:** [June 6, 2017, 10:54pm UTC](https://discuss.elastic.co/t/conditional-update-to-the-document/64964/17 "2017-06-06T22:54:12Z")
Have exactly the same issue, did anyone find some easy way to work around this, as I can't base it on versions. ---
**Author:** ![timothytong](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timothytong/32/24168_2.png) [@timothytong](https://discuss.elastic.co/u/timothytong)\ **Post date:** [November 14, 2017, 12:17am UTC](https://discuss.elastic.co/t/conditional-update-to-the-document/64964/18 "2017-11-14T00:17:49Z")
A clean way to do this without enumerating the entire map would be wrapping the entire map inside a wrapper object (e.g. called "document") then replace the original with `ctx._source = params.document`. A full example: ```auto POST /name/type/1/_update { "scripted_upsert": true, "script" : { "inline": "if (ctx._source.timestamp >= params.timestamp) { ctx.op = 'none' } else { ctx._source = params.document }", "lang": "painless", "params": { "document": { "timestamp": 112255 } } }, "upsert": { "timestamp": 33333 } } ``` Hope this helps ---
**Author:** ![gauthamr05](https://avatars.discourse-cdn.com/v4/letter/g/d2c977/32.png) [@gauthamr05](https://discuss.elastic.co/u/gauthamr05)\ **Post date:** [April 18, 2018, 9:46am UTC](https://discuss.elastic.co/t/conditional-update-to-the-document/64964/19 "2018-04-18T09:46:31Z")
Even I'm looking for a similar solution Did you find any ? ---
**Author:** ![mqchau](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mqchau/32/48189_2.png) [@mqchau](https://discuss.elastic.co/u/mqchau)\ **Post date:** [May 3, 2019, 6:20pm UTC](https://discuss.elastic.co/t/conditional-update-to-the-document/64964/20 "2019-05-03T18:20:17Z")
Hi all, I'm looking for the solution to this problem too and I think I got it. Background information of my situation: I have one search entity in Elastic where the document is compiled from multiple microservices. For example, the PropertyOwner object has some properties in microservice A, some addresses in microservice B. The goal is to search for PropertyOwner based on any of those information: property name or address. Each microservice handles its own way of indexing a partial document of PropertyOwner, the only thing they have to agree on is the document id, which is the UUID of PropertyOwner. Then the fields of the documents will not overlap on each other. The final document looks like this: { id: "Owner 12345", serviceA\_property\_names: ["Property 1", "Property 2"], serviceA\_timestamp: 123456, serviceB\_addresses: ["123 Golden Dr."] serviceB\_timestamp: 123789 } Now if 2 simultaneous user changes the address of owners in service B. We want to update the address in the document, only if the serviceB\_timestamp is lower than that of the new request. Here's how I do it: ``` POST /name/type/Owner+12345/_update { "scripted_upsert": true, "script": { "lang": "painless", "inline": "if (ctx._source.serviceB_timestamp < params.doc.serviceB_timestamp) { ctx._source.putAll(params.doc) }", "params": { "doc": { "serviceB_timestamp": 124000, "serviceB_addresses": ["567 Silver Rd."] } } } } ``` The method `_source.putAll` is from Java 11 documentation: [https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/Map.html#putAll(java.util.Map)](https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/Map.html#putAll(java.util.Map)) [Next page](https://discuss.elastic.co/t/conditional-update-to-the-document/64964.md?page=2)