[Java SDK] Optimistic lock occurs since switching to the 8.10 SDK

Elastic Stack Elasticsearch
1

Hello,

We were in the process of switching from Elasticsearch 7 to Elasticsearch 8 and the first step was to update all of our Java SDK from the version 7.17.9 to 8.10.9.

We have a CI that check regression on this part and what we encountered was a LOTS of version_conflict_engine_exception when doing bulk update or delete using the update by query or delete by query API.

When reverting to the old version, the problem disappear but we can't find difference on the HTTP request between the old version and the new version. (We have monitored http exchange to see if the query, the path or the body had impactful change but there was none)

On server side, we still use our ES 7.19 cluster and we plan to switch to ES 8 cluster as soon as we have validated the non-reg on our side.

We are a little bit lost about what happen so we need your help to see if this has happened to another people or if there is a breaking change we didn't see between both SDK version on these type of request.

Thanks for your help !

2

Here is the error log:

{
   "took":12,
   "timed_out":false,
   "total":1,
   "updated":0,
   "deleted":0,
   "batches":1,
   "version_conflicts":1,
   "noops":0,
   "retries":{
      "bulk":0,
      "search":0
   },
   "throttled_millis":0,
   "requests_per_second":-1.0,
   "throttled_until_millis":0,
   "failures":[
      {
         "index":"****",
         "type":"_doc",
         "id":"664327515f95f96f6e67c984",
         "cause":{
            "type":"version_conflict_engine_exception",
            "reason":"[664327515f95f96f6e67c984]: version conflict, required seqNo [160857], primary term [8]. current document has seqNo [160858] and primary term [8]",
            "index_uuid":"KC54qjZWSJq5J6fPzskedQ",
            "shard":"0",
            "index":"****"
         },
         "status":409
      }
   ]
}

I was able to reproduce with a simple script that update the same document multiple time per s.
Using the old SDK, there is no error, but using the new SDK, half of the request failed due to this error.

I will try to save a Wireshark so I can give you the network log to see if I'm missing something.