Version check on update?

Elastic Stack Elasticsearch
1

Sorry to repost an issuehttps://github.com/elasticsearch/elasticsearch/issues/3027,
but I thought I this might be seen by more people here:

I'm trying to use version checking with document updates using the Java
API, but I'm not sure how; there is no 'setVersion'
option. This is a feature supported by HTTP, as
http://www.elasticsearch.org/2011/02/08/versioning/ show an update
example with a version parameter.

Also, Just to clarify, when I read
http://www.elasticsearch.org/guide/reference/api/update/ :

"this operation still means full reindex of the document, it just removes
some network roundtrips and reduces chances of
version conflicts between the get and the index"

; I assume the 'get' and 'index' are between ES and lucene, rather than
between my client and ES? Otherwise I can't see
what roundtrips are saved.

Regards.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Hey,

you do not need to do anything, the version checking is done automatically
in case you are using the _update API. What happens is the following (very
abstract, not on source code level)

  1. Update request comes in for a document with an ID
  2. GET is done for the document (which includes the version)
  3. The update script is executed. Execution may last time X and returns a
    document
  4. The returned document is indexed under the known ID, the version from
    step 2 is used to make sure there was new indexation of the document
    between the GET ind the current index operation

All this happens inside of elasticsearch. It cannot happen on your client,
because you did not supply the whole document when sending the update
request. Also keep in mind that the time window between step 2 and 4 may
vary. For example with a really big document the update script execution
may take longer and so the window for another external process to reindex
is bigger.

Hope this helps. In case questions just ask

--Alex

On Tue, May 14, 2013 at 1:47 PM, Chris Hancock chs2048@gmail.com wrote:

Sorry to repost an issuehttps://github.com/elasticsearch/elasticsearch/issues/3027,
but I thought I this might be seen by more people here:

I'm trying to use version checking with document updates using the Java
API, but I'm not sure how; there is no 'setVersion'
option. This is a feature supported by HTTP, as
Elasticsearch Platform — Find real-time answers at scale | Elastic show an update
example with a version parameter.

Also, Just to clarify, when I read
Elasticsearch Platform — Find real-time answers at scale | Elastic :

"this operation still means full reindex of the document, it just removes
some network roundtrips and reduces chances of
version conflicts between the get and the index"

; I assume the 'get' and 'index' are between ES and lucene, rather than
between my client and ES? Otherwise I can't see
what roundtrips are saved.

Regards.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

This is version checking between ES and Lucene?
Some of the client operations (e.g. delete) support specifying a version
such that if the document doesn't match the version specified, the
operation is rejected
e.g If I want to delete a document, I might want to specify its version so
the wrong version (like a new, updated version created between the get and
the delete requests) isn't deleted.
I'd like to use the same functionality with updates, such that if the
document doesn't match the version I've specified, then ES rejects the
update. Is this possible?
Thanks for you time.

On Tuesday, 14 May 2013 13:36:52 UTC+1, Alexander Reelsen wrote:

Hey,

you do not need to do anything, the version checking is done automatically
in case you are using the _update API. What happens is the following (very
abstract, not on source code level)

  1. Update request comes in for a document with an ID
  2. GET is done for the document (which includes the version)
  3. The update script is executed. Execution may last time X and returns a
    document
  4. The returned document is indexed under the known ID, the version from
    step 2 is used to make sure there was new indexation of the document
    between the GET ind the current index operation

All this happens inside of elasticsearch. It cannot happen on your client,
because you did not supply the whole document when sending the update
request. Also keep in mind that the time window between step 2 and 4 may
vary. For example with a really big document the update script execution
may take longer and so the window for another external process to reindex
is bigger.

Hope this helps. In case questions just ask

--Alex

On Tue, May 14, 2013 at 1:47 PM, Chris Hancock <chs...@gmail.com<javascript:>

wrote:

Sorry to repost an issuehttps://github.com/elasticsearch/elasticsearch/issues/3027,
but I thought I this might be seen by more people here:

I'm trying to use version checking with document updates using the Java
API, but I'm not sure how; there is no 'setVersion'
option. This is a feature supported by HTTP, as
Elasticsearch Platform — Find real-time answers at scale | Elastic show an update
example with a version parameter.

Also, Just to clarify, when I read
Elasticsearch Platform — Find real-time answers at scale | Elastic :

"this operation still means full reindex of the document, it just removes
some network roundtrips and reduces chances of
version conflicts between the get and the index"

; I assume the 'get' and 'index' are between ES and lucene, rather than
between my client and ES? Otherwise I can't see
what roundtrips are saved.

Regards.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

Hi Chris,

Following up on Alexander's excellent overview, the following works well
enough for me. Wrap within a try/catch of course. The doc variable is a
reference to my own version of a parsed JSON document. The getter method
should be self-explanatory:

IndexRequestBuilder irb = client.prepareIndex(rec.getIndex(),
rec.getType(), rec.getId());

irb.setSource(rec.sourceToJson()).setOpType(IndexRequest.OpType.INDEX);
irb.setRefresh(refresh);

if (doc.hasVersion())
{
irb.setVersion(doc.getVersion());
if (doc.isVersionExternal())
irb.setVersionType(VersionType.EXTERNAL);
}

/* Set the TTL (time to live) if there is one */
TimeValue ttl = doc.getTTL();
if (ttl != null)
irb.setTTL(ttl.getMillis());

response = irb.execute().actionGet();

If the version type is INTERNAL (the default), then the supplied version
must match the version of the document as it exists in ES. If successful,
then the document in ES has its version number incremented.

If the version type is EXTERNAL, then the supplied version must be greater
than (not equal, and not less than) the version of the document as it
exists in ES. If successful, then the document in ES has its version number
set to the value you supplied.

Hope this helps!
Brian

On Tuesday, May 14, 2013 12:06:20 PM UTC-4, Chris Hancock wrote:

This is version checking between ES and Lucene?
Some of the client operations (e.g. delete) support specifying a version
such that if the document doesn't match the version specified, the
operation is rejected
e.g If I want to delete a document, I might want to specify its version so
the wrong version (like a new, updated version created between the get and
the delete requests) isn't deleted.
I'd like to use the same functionality with updates, such that if the
document doesn't match the version I've specified, then ES rejects the
update. Is this possible?
Thanks for you time.

On Tuesday, 14 May 2013 13:36:52 UTC+1, Alexander Reelsen wrote:

Hey,

you do not need to do anything, the version checking is done
automatically in case you are using the _update API. What happens is the
following (very abstract, not on source code level)

  1. Update request comes in for a document with an ID
  2. GET is done for the document (which includes the version)
  3. The update script is executed. Execution may last time X and returns a
    document
  4. The returned document is indexed under the known ID, the version from
    step 2 is used to make sure there was new indexation of the document
    between the GET ind the current index operation

All this happens inside of elasticsearch. It cannot happen on your
client, because you did not supply the whole document when sending the
update request. Also keep in mind that the time window between step 2 and 4
may vary. For example with a really big document the update script
execution may take longer and so the window for another external process to
reindex is bigger.

Hope this helps. In case questions just ask

--Alex

On Tue, May 14, 2013 at 1:47 PM, Chris Hancock chs...@gmail.com wrote:

Sorry to repost an issuehttps://github.com/elasticsearch/elasticsearch/issues/3027,
but I thought I this might be seen by more people here:

I'm trying to use version checking with document updates using the Java
API, but I'm not sure how; there is no 'setVersion'
option. This is a feature supported by HTTP, as
Elasticsearch Platform — Find real-time answers at scale | Elastic show an update
example with a version parameter.

Also, Just to clarify, when I read
Elasticsearch Platform — Find real-time answers at scale | Elastic :

"this operation still means full reindex of the document, it just
removes some network roundtrips and reduces chances of
version conflicts between the get and the index"

; I assume the 'get' and 'index' are between ES and lucene, rather than
between my client and ES? Otherwise I can't see
what roundtrips are saved.

Regards.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

Thanks, I appreciate your replies.

At the moment I'm using update to do partial updates so I don't have to
send the the entire document, and the update request builder doesn't have a
'setVersion' method.

Regards.

On Tuesday, 14 May 2013 17:42:41 UTC+1, InquiringMind wrote:

Hi Chris,

Following up on Alexander's excellent overview, the following works well
enough for me. Wrap within a try/catch of course. The doc variable is a
reference to my own version of a parsed JSON document. The getter method
should be self-explanatory:

IndexRequestBuilder irb = client.prepareIndex(rec.getIndex(),
rec.getType(), rec.getId());

irb.setSource(rec.sourceToJson()).setOpType(IndexRequest.OpType.INDEX);
irb.setRefresh(refresh);

if (doc.hasVersion())
{
irb.setVersion(doc.getVersion());
if (doc.isVersionExternal())
irb.setVersionType(VersionType.EXTERNAL);
}

/* Set the TTL (time to live) if there is one */
TimeValue ttl = doc.getTTL();
if (ttl != null)
irb.setTTL(ttl.getMillis());

response = irb.execute().actionGet();

If the version type is INTERNAL (the default), then the supplied version
must match the version of the document as it exists in ES. If successful,
then the document in ES has its version number incremented.

If the version type is EXTERNAL, then the supplied version must be greater
than (not equal, and not less than) the version of the document as it
exists in ES. If successful, then the document in ES has its version number
set to the value you supplied.

Hope this helps!
Brian

On Tuesday, May 14, 2013 12:06:20 PM UTC-4, Chris Hancock wrote:

This is version checking between ES and Lucene?
Some of the client operations (e.g. delete) support specifying a version
such that if the document doesn't match the version specified, the
operation is rejected
e.g If I want to delete a document, I might want to specify its version
so the wrong version (like a new, updated version created between the get
and the delete requests) isn't deleted.
I'd like to use the same functionality with updates, such that if the
document doesn't match the version I've specified, then ES rejects the
update. Is this possible?
Thanks for you time.

On Tuesday, 14 May 2013 13:36:52 UTC+1, Alexander Reelsen wrote:

Hey,

you do not need to do anything, the version checking is done
automatically in case you are using the _update API. What happens is the
following (very abstract, not on source code level)

  1. Update request comes in for a document with an ID
  2. GET is done for the document (which includes the version)
  3. The update script is executed. Execution may last time X and returns
    a document
  4. The returned document is indexed under the known ID, the version from
    step 2 is used to make sure there was new indexation of the document
    between the GET ind the current index operation

All this happens inside of elasticsearch. It cannot happen on your
client, because you did not supply the whole document when sending the
update request. Also keep in mind that the time window between step 2 and 4
may vary. For example with a really big document the update script
execution may take longer and so the window for another external process to
reindex is bigger.

Hope this helps. In case questions just ask

--Alex

On Tue, May 14, 2013 at 1:47 PM, Chris Hancock chs...@gmail.com wrote:

Sorry to repost an issuehttps://github.com/elasticsearch/elasticsearch/issues/3027,
but I thought I this might be seen by more people here:

I'm trying to use version checking with document updates using the Java
API, but I'm not sure how; there is no 'setVersion'
option. This is a feature supported by HTTP, as
Elasticsearch Platform — Find real-time answers at scale | Elastic show an update
example with a version parameter.

Also, Just to clarify, when I read
Elasticsearch Platform — Find real-time answers at scale | Elastic :

"this operation still means full reindex of the document, it just
removes some network roundtrips and reduces chances of
version conflicts between the get and the index"

; I assume the 'get' and 'index' are between ES and lucene, rather than
between my client and ES? Otherwise I can't see
what roundtrips are saved.

Regards.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

6

Hi Chris,

Now I see what you are trying to do. Yes, the UpdateRequestBuilder has no
setVersion, but it does have a retry count for version errors. I guess I
would need to spend a few days exploring this class's actual behavior to
see what it really is all about.

Since I always store the source, and I haven't seen a way for Elasticsearch
to surgically modify just a few fields in the source during an update, I
always issue either a create (to ensure an _id doesn't already exist) or an
index request. Since the source appears atomic anyway, I have my own
higher-level document class that can be loaded from JSON, then updated
(one, two, or more fields), and then re-emitted to JSON. So an
IndexRequestBuilder makes sense to me, and its setVersion and
setVersionType methods work very well and have become clear and easy to use.

Of course, this means I need the response from a query/filter or get-by-id
to get the current document on which to perform an update, but that's a
typical use case anyway. And then I can extract the version and pass it in
to the IndexRequestBuilder and perform my own re-try (re-get and re-update)
as needed for version conflicts and particular use case.

Sorry I couldn't have been more help in your situation. I hope you get the
information you are looking for, and I will follow this thread in the hopes
I will learn something more!

Regards,
Brian

On Tuesday, May 14, 2013 1:49:14 PM UTC-4, Chris Hancock wrote:

Thanks, I appreciate your replies.

At the moment I'm using update to do partial updates so I don't have to
send the the entire document, and the update request builder doesn't have a
'setVersion' method.

Regards.

On Tuesday, 14 May 2013 17:42:41 UTC+1, InquiringMind wrote:

Hi Chris,

Following up on Alexander's excellent overview, the following works well
enough for me. Wrap within a try/catch of course. The doc variable is a
reference to my own version of a parsed JSON document. The getter method
should be self-explanatory:

IndexRequestBuilder irb = client.prepareIndex(rec.getIndex(),
rec.getType(), rec.getId());

irb.setSource(rec.sourceToJson()).setOpType(IndexRequest.OpType.INDEX);
irb.setRefresh(refresh);

if (doc.hasVersion())
{
irb.setVersion(doc.getVersion());
if (doc.isVersionExternal())
irb.setVersionType(VersionType.EXTERNAL);
}

/* Set the TTL (time to live) if there is one */
TimeValue ttl = doc.getTTL();
if (ttl != null)
irb.setTTL(ttl.getMillis());

response = irb.execute().actionGet();

If the version type is INTERNAL (the default), then the supplied version
must match the version of the document as it exists in ES. If successful,
then the document in ES has its version number incremented.

If the version type is EXTERNAL, then the supplied version must be
greater than (not equal, and not less than) the version of the document as
it exists in ES. If successful, then the document in ES has its version
number set to the value you supplied.

Hope this helps!
Brian

On Tuesday, May 14, 2013 12:06:20 PM UTC-4, Chris Hancock wrote:

This is version checking between ES and Lucene?
Some of the client operations (e.g. delete) support specifying a version
such that if the document doesn't match the version specified, the
operation is rejected
e.g If I want to delete a document, I might want to specify its version
so the wrong version (like a new, updated version created between the get
and the delete requests) isn't deleted.
I'd like to use the same functionality with updates, such that if the
document doesn't match the version I've specified, then ES rejects the
update. Is this possible?
Thanks for you time.

On Tuesday, 14 May 2013 13:36:52 UTC+1, Alexander Reelsen wrote:

Hey,

you do not need to do anything, the version checking is done
automatically in case you are using the _update API. What happens is the
following (very abstract, not on source code level)

  1. Update request comes in for a document with an ID
  2. GET is done for the document (which includes the version)
  3. The update script is executed. Execution may last time X and returns
    a document
  4. The returned document is indexed under the known ID, the version
    from step 2 is used to make sure there was new indexation of the document
    between the GET ind the current index operation

All this happens inside of elasticsearch. It cannot happen on your
client, because you did not supply the whole document when sending the
update request. Also keep in mind that the time window between step 2 and 4
may vary. For example with a really big document the update script
execution may take longer and so the window for another external process to
reindex is bigger.

Hope this helps. In case questions just ask

--Alex

On Tue, May 14, 2013 at 1:47 PM, Chris Hancock chs...@gmail.comwrote:

Sorry to repost an issuehttps://github.com/elasticsearch/elasticsearch/issues/3027,
but I thought I this might be seen by more people here:

I'm trying to use version checking with document updates using the
Java API, but I'm not sure how; there is no 'setVersion'
option. This is a feature supported by HTTP, as
Elasticsearch Platform — Find real-time answers at scale | Elastic show an update
example with a version parameter.

Also, Just to clarify, when I read
Elasticsearch Platform — Find real-time answers at scale | Elastic :

"this operation still means full reindex of the document, it just
removes some network roundtrips and reduces chances of
version conflicts between the get and the index"

; I assume the 'get' and 'index' are between ES and lucene, rather
than between my client and ES? Otherwise I can't see
what roundtrips are saved.

Regards.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

7

Just to make it clear, is it worth re-opening the issue, which refers to
index requests, not updates.

On Tuesday, 14 May 2013 12:47:05 UTC+1, Chris Hancock wrote:

Sorry to repost an issuehttps://github.com/elasticsearch/elasticsearch/issues/3027,
but I thought I this might be seen by more people here:

I'm trying to use version checking with document updates using the Java
API, but I'm not sure how; there is no 'setVersion'
option. This is a feature supported by HTTP, as
Elasticsearch Platform — Find real-time answers at scale | Elastic show an update
example with a version parameter.

Also, Just to clarify, when I read
Elasticsearch Platform — Find real-time answers at scale | Elastic :

"this operation still means full reindex of the document, it just removes
some network roundtrips and reduces chances of
version conflicts between the get and the index"

; I assume the 'get' and 'index' are between ES and lucene, rather than
between my client and ES? Otherwise I can't see
what roundtrips are saved.

Regards.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

8

Hi Brian

On 14 May 2013 21:44, InquiringMind brian.from.fl@gmail.com wrote:

Since I always store the source, and I haven't seen a way for
Elasticsearch to surgically modify just a few fields in the source during
an update,

You can do this by using the update API with a script:

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

9

Hi Chris

The update API is intended for making idempotent changes. If you already
have the document and the version, why not just use the index API? I
realise it means resending the whole _source again, but that's essentially
what happens with update anyway (just internally)

Clint

On 17 May 2013 16:47, Chris Hancock chs2048@gmail.com wrote:

Just to make it clear, is it worth re-opening the issue, which refers to
index requests, not updates.

On Tuesday, 14 May 2013 12:47:05 UTC+1, Chris Hancock wrote:

Sorry to repost an issuehttps://github.com/elasticsearch/elasticsearch/issues/3027,
but I thought I this might be seen by more people here:

I'm trying to use version checking with document updates using the Java
API, but I'm not sure how; there is no 'setVersion'
option. This is a feature supported by HTTP, as
Elasticsearch Platform — Find real-time answers at scale | Elastichttp://www.elasticsearch.org/2011/02/08/versioning/show an update
example with a version parameter.

Also, Just to clarify, when I read http://www.elasticsearch.org/**
guide/reference/api/update/http://www.elasticsearch.org/guide/reference/api/update/:

"this operation still means full reindex of the document, it just removes
some network roundtrips and reduces chances of
version conflicts between the get and the index"

; I assume the 'get' and 'index' are between ES and lucene, rather than
between my client and ES? Otherwise I can't see
what roundtrips are saved.

Regards.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

10

Hi Brian, thanks for the reply.

I don't already have the document, but I know what fields I want to change,
and what to change them to - I want to leave all others fields alone. I'm
sending this request over the network, whereas elasticsearch and lucene are
on the same box, so I'd rather reduce traffic between the java client and
elasticsearch. It increases traffic to send a get request for the document,
and then an index request for the entire document. I'd prefer to just do a
partial update, and never have to get the document, or send any other field
than those that change. I can't do this safely if I cannot specify the
expected document version, such that ES will reject the update if the
version is different (as it is with deletes, for example). I'm not sure why
update objects don't support this.

On Saturday, 18 May 2013 10:12:59 UTC+1, Clinton Gormley wrote:

Hi Chris

The update API is intended for making idempotent changes. If you already
have the document and the version, why not just use the index API? I
realise it means resending the whole _source again, but that's essentially
what happens with update anyway (just internally)

Clint

On 17 May 2013 16:47, Chris Hancock <chs...@gmail.com <javascript:>>wrote:

Just to make it clear, is it worth re-opening the issue, which refers to
index requests, not updates.

On Tuesday, 14 May 2013 12:47:05 UTC+1, Chris Hancock wrote:

Sorry to repost an issuehttps://github.com/elasticsearch/elasticsearch/issues/3027,
but I thought I this might be seen by more people here:

I'm trying to use version checking with document updates using the Java
API, but I'm not sure how; there is no 'setVersion'
option. This is a feature supported by HTTP, as
Elasticsearch Platform — Find real-time answers at scale | Elastichttp://www.elasticsearch.org/2011/02/08/versioning/show an update
example with a version parameter.

Also, Just to clarify, when I read http://www.elasticsearch.org/**
guide/reference/api/update/http://www.elasticsearch.org/guide/reference/api/update/:

"this operation still means full reindex of the document, it just
removes some network roundtrips and reduces chances of
version conflicts between the get and the index"

; I assume the 'get' and 'index' are between ES and lucene, rather than
between my client and ES? Otherwise I can't see
what roundtrips are saved.

Regards.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

11

Hi Chris

On 18 May 2013 17:59, Chris Hancock chs2048@gmail.com wrote:

I don't already have the document, but I know what fields I want to
change, and what to change them to - I want to leave all others fields
alone. I'm sending this request over the network, whereas elasticsearch and
lucene are on the same box, so I'd rather reduce traffic between the java
client and elasticsearch. It increases traffic to send a get request for
the document, and then an index request for the entire document. I'd prefer
to just do a partial update, and never have to get the document, or send
any other field than those that change. I can't do this safely if I cannot
specify the expected document version, such that ES will reject the update
if the version is different (as it is with deletes, for example). I'm not
sure why update objects don't support this.

You really need to read the documentation on the update API. It does
exactly what you need, including ensuring that no data is lost by parallel
updates. See the retry_on_conflict parameter:

clint

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

12

@Clinton:
Doesn't 'retry_on_conflict' apply to the connection between ES and lucene?
The update API describes a http interface, whereas I am using the java API,
and cannot see any method on update requests that allow me to pass a
version.

On 18 May 2013 18:15, Clinton Gormley clint@traveljury.com wrote:

Hi Chris

On 18 May 2013 17:59, Chris Hancock chs2048@gmail.com wrote:

I don't already have the document, but I know what fields I want to
change, and what to change them to - I want to leave all others fields
alone. I'm sending this request over the network, whereas elasticsearch and
lucene are on the same box, so I'd rather reduce traffic between the java
client and elasticsearch. It increases traffic to send a get request for
the document, and then an index request for the entire document. I'd prefer
to just do a partial update, and never have to get the document, or send
any other field than those that change. I can't do this safely if I cannot
specify the expected document version, such that ES will reject the update
if the version is different (as it is with deletes, for example). I'm not
sure why update objects don't support this.

You really need to read the documentation on the update API. It does
exactly what you need, including ensuring that no data is lost by parallel
updates. See the retry_on_conflict parameter:
Elasticsearch Platform — Find real-time answers at scale | Elastic

clint

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/M49krcDGPHc/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

13

Chris - you don't need to specify a version. Elasticsearch retrieves the
current document, applies your update and reindexes the document, checking
that the version hasn't been incremented in the meantime. If it has been
incremented, then it retries the process retry_on_conflict times (which
defaults to 0) and either ends up succeeding, or returning a conflict error
to you.

You said:

I don't already have the document, but I know what fields I want to
change, and what to change them to - I want to leave all others fields
alone.

Right - that's exactly what the update api does.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

14

Ah, so this is what you mean by idempotent changes - I wanted to prevent
updates when another user had already made a change to that document (to
ensure this change was preserved somehow), but I guess this isn't supported?

Thanks for your help.

On 20 May 2013 12:24, Clinton Gormley clint@traveljury.com wrote:

Chris - you don't need to specify a version. Elasticsearch retrieves the
current document, applies your update and reindexes the document, checking
that the version hasn't been incremented in the meantime. If it has been
incremented, then it retries the process retry_on_conflict times (which
defaults to 0) and either ends up succeeding, or returning a conflict error
to you.

You said:

I don't already have the document, but I know what fields I want to
change, and what to change them to - I want to leave all others fields
alone.

Right - that's exactly what the update api does.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/M49krcDGPHc/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.