Document is not updated although the response is successful

Elastic Stack Elasticsearch
1

Hey,

I am having a weird scenario, in which sometimes Elasticsearch claims to update a document successfully but it actually didn't do any thing.

So this is my code:

  override def addTerms(id: String, terms: Set[String])) = {
    val addTermsScript =
      s"""if (ctx._source.names == null)
          |   ctx._source.names=newItems
          |else
          |   ctx._source.names<< newItems;
          |ctx._source.names = ctx._source.names.flatten().unique();""".stripMargin
    client.execute(update id id in indexName / documentType retryOnConflict RETRIES script {
      script(addTermsScript).params(Map("newItems" -> terms.toArray))
    }).map(res => {
      Logger.info(s"Add terms: $terms response on id: $id - Shared info: ${res.getShardInfo}")
    })
  }

The response was :
Add terms: Set("term1") response on id: some_id - Shared info: "_shards"{"total":2,"successful":1,"failed":0}

But when looking on the document, the field was not updated. This code is working, but once in a while I get this false update.

Am I looking on the correct portion of the response?
Any idea?

2

Hi @tpraizler,

I kind of get your Scala code but let's take Scala out of the question here and take step back. Here is a complete example that you can run in Sense:

DELETE /update-test

PUT /update-test/type1/1
{
    "items" : ["foo"]
}

If we retrieve the document now with GET /update-test/type1/1 the array will contain just "foo". Notice that the version field says "1":

{
   "_index": "update-test",
   "_type": "type1",
   "_id": "1",
   "_version": 1,
   "forced_refresh": false,
   "_shards": {
      "total": 2,
      "successful": 1,
      "failed": 0
   },
   "created": true
}

Now we do an update (script slightly modified from your version):

POST /update-test/type1/1/_update
{
   "script": {
      "inline": "ctx._source.items = (ctx._source.items == null) ? newItems : ctx._source.items + newItems",
      "lang": "groovy",
      "params": {
         "newItems": [
            "bar",
            "baz"
         ]
      }
   }
}

We get as response:

{
   "_index": "update-test",
   "_type": "type1",
   "_id": "1",
   "_version": 2,
   "forced_refresh": false,
   "_shards": {
      "total": 2,
      "successful": 1,
      "failed": 0
   }
}

Am I looking on the correct portion of the response?

The important bit in the response is the _version field which shows that the document's version has been updated.

Daniel

3

Thanks @danielmitterdorfer! Very helpful!

In my app, at the point where the above method is invoked, I don't have the version of the current document in elasticsearch.
I don't want to read the document before update (performance penalty) \ add version to this class coupling it to elasticsaerch.

Do you have any other suggestion on how to handle this?
Is it possible for an update request to do nothing, and return the current version of the document?

Thanks!

4

Hi @tpraizler,

In my app, at the point where the above method is invoked, I don't have the version of the current document in elasticsearch.

I just showed this for demonstration purposes. Actually, the update request should just be fire-and-forget. The only thing that I'd check is that "failed" is 0. I am not sure how you determined that some documents were not updated. Do you have a very long refresh interval maybe? (you can also force a refresh on update with a parameter but this will have performance implications so I'd just do this for testing!).

Is it possible for an update request to do nothing, and return the current version of the document?

Yes, this is possible. This will return the current document as is:

POST /update-test/type1/1/_update
{
    "doc": { }
}

(but you can achieve the same by just retrieving the document ;)).

You can even avoid the update in a scripted update, by specifying ctx.op = \"none\"(see docs).

Daniel

1 Like
5

Thanks.

Amm, in my case I update a document (with the method in the first post), and get failed : 0.
But the document was not updated.

This is my problem, the update does not fail, but does not update anything.
It happens once in a while, not every update. sounds like a bug?

anything else beside the failed property that might indicate if the update was successful?

6

Hi @tpraizler,

This is my problem, the update does not fail, but does not update anything.

How do you verify that? The updated document will not appear in search results until the next refresh (that's why I've asked about your refresh interval).

anything else beside the failed property that might indicate if the update was successful?

I'd log the document id and the version that is returned in every response if you really suspect a problem. This should help you to understand whether the document has been updated (I understand that you cannot use this as a kind of post-condition check in your application but it should at least get you started in debugging the problem I hope).

Daniel

7

Thanks @danielmitterdorfer.

My refresh interval is 1 second (the default).
And I verified it by querying this document manually using sense (The best verification..:slight_smile: )

So I will need to log the version when I get the document, and also when update that document? meaning the 2 operations should be logged?

8

Hi @tpraizler,

My refresh interval is 1 second (the default).
And I verified it by querying this document manually using sense (The best verification.. )

Ok, that rules out a lot of problems. :slight_smile:

So I will need to log the version when I get the document, and also when update that document?

Provided that this is the only code path that updates these documents you just need to log after the response of the update. I imagine that your log file looks something like:

doc id [17] updated to version [3]
doc id [18] updated to version [2]
[...]
doc id [17] updated to version [3]

The expectation is that all lines are unique. If you see something like the last line in the example log then you know that something is odd and you need to dig deeper.

Daniel

9

I see! Make sense!
I will do that and update if I find anything.

Thanks a lot @danielmitterdorfer

10

You're welcome. :slight_smile: