Is version conflict bulk write error (409) only occur when I'm updating the same document from 2 different writers?
I am starting to see this error more often in our system and trying to figure out if our data pattern has changed or it's something else.
The other possibility is ES7.15 raised this to error level comparing to ES6 so I'm seeing it more now?
That's a likely cause, yes.
thanks.
Another follow up on this.
If I don't care about preserving old data. I just overwrite.
What caused ES to have the concept of "version". It's first come first serve for my case (last write wins).
Why does ES throw error 409? What's the trigger point?
For example:
If say I have 10 writers. Each writer writes to the same docID repeatedly with random delay.
Writer1 writes A, writer2 writes B, etc.
Should I ever get error 409 in this scenario?
*note: I should say "sequence number" instead of version. That's what the error log states.
If I remember correctly Elasticsearch will first read the document from the shard during an update and then perform the update and write it again. When doing this it will verify that the document has not been changed by checking the version number when writing and this is what leads to conflict with frequent updates. As far as I know there is no way to completely disable this behaviour.
Elasticsearch has as far as I know never been optimized for frequent updates, which in addition to the issues mentioned above also can result in a lot of small segments which is inefficient and can cause a lot of overhead and be slow.
This sounds more like ES internal timing issue, am I right?
Multiple threads writing to the same docID but somehow later thread happens to get the CPU to perform the update first, hence sequence unexpected.
It's not my application's logic.
*We typically will shard the application so one writer will write to the same docID, etc. But not always the case.
If you have updates to the same document happening in parallel, two threads can read the document in parallel, perform the update and only one of them will then succeed as the other one will notice a changed version.
ah, I think we should've use "index" api instead of update since we are simply replacing.
Thanks.
I think that will result in the same behaviour if you specify an existing document id. It is still an update.
Oh, really. hmmmm.
The documentation seems to suggest updateRequest() is read->modify->write (which supports partial update).
If I only need to replace document, just use indexRequest().
I am running 7.15. This post is for 8.4. Maybe it's not applicable?
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.