Disable increment of version counter on some update operations possible?

Elastic Stack Elasticsearch
1

Is it possible to disable the increment of the version counter on some
update operations?

I've a views counter which is updated whenever user "opens" a file in ma
app.
body: { script: 'ctx._source.views += 1',}

I don't want this to affect the version number, cause when some other user
is doing updates on other document attributes, updates could not be saved
due to version conflict.
On the other hand I'd like to use the versioning feature to make sure
multiple users are overriding their changes. (except from the views field)

Thanks, Joa

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ea2893c7-464e-4de3-ac5a-01520eb293dd%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Not sure I fully understand the use case but AFAIK you can not keep the same version.

If needed, you can try to manage version by yourself using external versioning: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-index_.html#index-versioning

Note sure it helps though.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 17 janvier 2014 at 14:13:19, joa (joafeldmann@gmail.com) a écrit:

Is it possible to disable the increment of the version counter on some update operations?

I've a views counter which is updated whenever user "opens" a file in ma app.
body: { script: 'ctx._source.views += 1',}

I don't want this to affect the version number, cause when some other user is doing updates on other document attributes, updates could not be saved due to version conflict.
On the other hand I'd like to use the versioning feature to make sure multiple users are overriding their changes. (except from the views field)

Thanks, Joa

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ea2893c7-464e-4de3-ac5a-01520eb293dd%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.52d9456f.4f4ef005.dc5%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/groups/opt_out.

3

Joa,

To allow an existing document to be updated, replaced, or deleted,
ElasticSearch implements the following behaviors:

  1. Internal version numbers for a document start at 1 and increment by one
    after each update to a document. When indexing or deleting an existing
    document, an internal version number must match the document's version or
    the operation will be rejected.

  2. External version numbers for a document start at whatever value is
    specified by the application. When indexing or deleting an existing
    document, an external version number must be greater than the document's
    version or the operation will be rejected. When the operation completes
    successfully, the document's version will be the externally specified
    version number.

So, to safely update part of a document, the best approach that comes to
mind is to use internal versioning and then:

  1. When you have the existing document you wish to update, keep its
    original contents and its version number: current-doc, let's say.

  2. Keep your updates in a separate document: updated-doc, let's say.

  3. Merge updated-doc into current-doc to generate a new document. new-doc,
    let's say. It has the same version number as current-doc.

  4. If the update of new-doc fails with a version check, then re-read the
    current document (saving its new version number) into current-doc, and
    repeat step 3 until success or some configured limit.

In essence, it's a very nice way to let versioning ensure that updates
across different sources are non-destructively merged. ES rocks!

Does this help?

Brian

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6b0c7d35-b71a-41e6-a95f-8b69d4798e3c%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

David and Brian, thanks for you answers. Brian, that might work, but its a
lot of code for a simple counter update.
I hoped that there would be a flag like keepVersionNumber = true as
parameter when updating.

Do you think this might be a reasonable feature request or does it make no
sense at all?

On Friday, January 17, 2014 4:15:23 PM UTC+1, InquiringMind wrote:

Joa,

To allow an existing document to be updated, replaced, or deleted,
Elasticsearch implements the following behaviors:

  1. Internal version numbers for a document start at 1 and increment by one
    after each update to a document. When indexing or deleting an existing
    document, an internal version number must match the document's version or
    the operation will be rejected.

  2. External version numbers for a document start at whatever value is
    specified by the application. When indexing or deleting an existing
    document, an external version number must be greater than the document's
    version or the operation will be rejected. When the operation completes
    successfully, the document's version will be the externally specified
    version number.

So, to safely update part of a document, the best approach that comes to
mind is to use internal versioning and then:

  1. When you have the existing document you wish to update, keep its
    original contents and its version number: current-doc, let's say.

  2. Keep your updates in a separate document: updated-doc, let's say.

  3. Merge updated-doc into current-doc to generate a new document. new-doc,
    let's say. It has the same version number as current-doc.

  4. If the update of new-doc fails with a version check, then re-read the
    current document (saving its new version number) into current-doc, and
    repeat step 3 until success or some configured limit.

In essence, it's a very nice way to let versioning ensure that updates
across different sources are non-destructively merged. ES rocks!

Does this help?

Brian

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b98cdc35-b66e-491a-af35-5cc0c092c924%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

Not a lot of code at all. Just some code that will be a lot of fun to
write. I did it with a lot more code, before I got into Jackson data
binding. But I wanted a generic Document class that adapted to a schema;
you won't need all that.

First, get Jackson and add the following jars to your classpath:

jackson-annotations-2.2.3.jar
jackson-core-2.2.3.jar
jackson-databind-2.2.3.jar

Create a class. Let's call it Document. Using Jackson's data binding is
dirt-easy and tons of fun... and super fast too.

Query for your document, and using Jackson's data binding, deserialize its
getSourceAsString into your Document just like that: Poof! Then add the
index, type, id, and version for good measure. A pittance of code.

Write a merge method to merge a source Document into a target Document.
Again, very simple. Your setters might need a boolean for each internal
data member to indicate it it's set or not. Or treat a null String or 0
numeric value as unset. Whatever fits your needs.

Then your Document.toString method could be @Override to marshall your
Document to a JSON string that matches the _source.

The remaining code would be to get-by-id (GetRequest) and issue an index
request.

I don't think it would make sense to build this into Elasticsearch; ES
doesn't know about your documents and what a merge might mean, what field
update rules you might require, or even what fields and their types exist
in your documents. And suppressing the versioning would then defeat
Elasticsearch's ability to detect when one update is trashing the changes
made by another update. Without the ability to lock a document and wrap
updates and queries with an ACID transaction (which ES won't ever pollute
itself with), the existing (and really awesome) versioning behavior gives
you everything you need to do the job much more easily.

And Elasticsearch along with adding in Jackson data binding make writing it
yourself an effort that will bring joy and success to you after perhaps a
day or two. Well, it did for me!

Have fun!

Brian

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7633d955-0fb5-49b5-881c-03dde3c3aa99%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

6

Also, see
https://groups.google.com/d/topic/elasticsearch/1152jRW_rQg/discussion for
how easy it is to issue an index request in Java. Hardly any code at all.
(Keep in mind the method name mismatch between the actual code and the
on-line docs, but those issues are clearly shown in this post).

And
http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/index_.html#beanscontains an example of how easy it is to write the code to covert between a
Java object and a JSON string.

Then Google for the annotations you'll need to customize Jackson data
binding for your document class. And if you run into any data binding
questions related to this, feel free to ask me.

Brian

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2ed979e7-cb01-4f48-a670-68288d36d06d%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

7

Thanks for your effort, Brian.
I'll think about this (I'm working with node.js not with Java anyway), but
I already opened an issue for this
(https://github.com/elasticsearch/elasticsearch/issues/4791).

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5b75b820-53c4-440c-8144-5427381f21e9%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

8

You can use external versioning, no need for an issue.

Jörg

On Mon, Jan 20, 2014 at 11:58 AM, joa joafeldmann@gmail.com wrote:

Thanks for your effort, Brian.
I'll think about this (I'm working with node.js not with Java anyway), but
I already opened an issue for this (
Possibility to disable the version increment on some updates · Issue #4791 · elastic/elasticsearch · GitHub).

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH2%2BT2VxLGuQy75Vxqr%2BPPM_DazVf6FSJSCpZcwVsxf1w%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

9

Hi Jörg,

as I understand, even with external versioning its not possible to update a
doc without changing/not incrementing the version number at all.

  1. User A loads DOC123 with version 20 and locally starts editing critical
    fields
  2. User B simply loads DOC123 to view/read only. A view counter will be
    incremented, so also the version number will be set to 21 (or something
    higher with external versioning, but not to 20 again)
  3. User A tries to send the updates from (1) with version number 20 to
    ensure he has the current version and update will fail, cause version
    number has changed

Thanks
Joa

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b691324b-7955-4379-b014-ee9d27a29e38%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

10

It is possible, ES does not increment the version if the version type is
external.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoG8isRj148fgyDUbL1G10a2gUoJSSWd5j63uFdPb1QPVQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

11

Sorry, I don't get it :wink:
Even when I set the version_type to external I also need to pass a version
number greater the current doc version or my update will fail, right?

Index:
curl -XPUT
'http://localhost:9200/videos/video/1?version_type=external&version=100' -d
'{
"name": "My video",
"viewCounter": 0
}'

Update/Increment the viewCounter: I need to pass (minimum) 101, otherwise
updating will fail ('m using current master to support version params on
_update, see https://github.com/elasticsearch/elasticsearch/issues/3684)
curl -XPOST
'localhost:9200/videos/video/1/_update?version_type=external&version=101'
-d '{
"script" : "ctx._source.viewCounter += 1"
}'

So it is not possible to increment the viewCounter without changing the
version number either internal or external. One solution would be to copy
the doc and reindex like Brian suggested above?
Therefore I thought it would be nice to pass a parameter to _update to tell
ES to not change the version number...

Thanks four your help!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/df637087-4c49-49e9-b1c5-1731843b9295%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

12

Are there any reasons for using versioning at all if you want to keep the
same version? If you just drop versioning in your use case, does it make a
difference?

I just learned from the latest source that internal and external versioning
have divergent semantics in the update action conflict detection: while
internal versioning is matching for unequality, external versioning matches
for a greater value.

Could be a small bug, introduced in

Anyway, I think the version should change if versioning is used. Otherwise
it won't make no sense at all.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoG47gvgxgUrT_wAaR%3D31ue9LVYE3wrQgcyt526aV16Brg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

13

I'd really like to use the versioning feature to ensure, that updates on
critical fields will be done on the latest version. (Back to my video
example, this could be changes on video access permissions or something).

But when I am updating not critical fields I'd like the version to remain
unchanged to not collide with critical updates done in parallel.
So a parameter passed with _update could versioning make optional.

I also described it
here: https://github.com/elasticsearch/elasticsearch/issues/4791

Joa

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3fcd482d-a236-4cd1-aef5-82a17ada8162%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

14

I don't understand your use case.

By not checking or incrementing the version number, one user's changes
could obliterate the changes made to those or even to other fields by other
users in between that first users's get of the document and that users's
subsequent version-less update. You will have a race condition in which
some users' changes will occasionally disappear.

The version handling steps I mentioned are easy in Java. And since the
_source is JSON, and JSON is "JavaScript Object Notation", I should strong
suspect that the equivalent JavaScript is similarly straight-forward.

Brian

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/16a9a32d-28fa-4aa6-9349-89d040ed6a04%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

15

Not when doing partial updates on the doc. I am always just updating the
fields that have changed.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/59019ced-6c8a-44d5-ac8c-9c1fa728349c%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

16

There is no such thing as a partial update on the doc. Even if you use the
update API, it still deletes the old document and indexes an entirely new
document. Version numbers are essential for the internal workings of
Elasticsearch. You cannot disable them.

And as you found, external versions must also always be higher than the
previous version, otherwise the write operation will fail (as they should).

The only way to do this is to retry the update, which you can do
automatically with the update API

On 20 January 2014 18:53, joa joafeldmann@gmail.com wrote:

Not when doing partial updates on the doc. I am always just updating the
fields that have changed.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/59019ced-6c8a-44d5-ac8c-9c1fa728349c%40googlegroups.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPt3XKQbkMtzKd55FaQ%2BfdgY%3DQ_VChhnOZp_DHKmgSN9%2Bq82fQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

17

Thanks Clinton

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e11afdd7-9ee8-4038-b382-e12f6d6dbdfb%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.