Conditional updates with Update API

Elastic Stack Elasticsearch
1

Here's the situation I'm trying to solve for:

I have documents getting upserted by multiple processes using the Bulk API. They look like this:

{
   "ver": 1,
   "foo": 3
}
{
   "ver": 2,
   "foo": 4
}
{
   "ver": 4,
   "foo": 5
}
{
   "ver": 3,
   "foo": 4
}

I would like this final state of ES to have:

{
   "ver": 4,
   "foo": 5
}

Because they are coming through multiple queues, they are not guaranteed to be in order. What I would like is for the update to only occur if "ver" is higher than the one currently stored.

Approaches I've considered:

  1. Use versioning support to supply the previous version number such that the update only succeeds if this is the next sequential document. Problems with this: if one of the updates is missing from the stream, later updates would never be written.

  2. Use version_type=external. This seems like exactly what we want, but it is not supported by the Update API.

  3. Using an inline script. Problem here as I understand it is then needing to manual handle every field of the document in the script, since you can't have both a script & doc in the update.

Any suggestions?

2

"External" versioning is the correct solution. Use index API Index API | Elasticsearch Guide [8.11] | Elastic

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.