Conditional insert of data in ES


(Amit Soni) #1

Hi everyone - I am wondering if ElasticSearch provides a mechanism wherein
an insert/update of a document is allowed only if a specified condition is
met.
Lets say that I have a document with id=id1 and my_version=8. Now an update
to this document should be allowed only if the field my_version has a value
greater than 8. If not, the update should not be done.

For my use case, I have more than one applications (message bus) entering
data into ES server and I want to ensure that an existing copy of the
document does not get overwritten with a stale copy, hence I have a
versioning system of my own.

Does ElasticSearch provide a way to do this?

-Amit.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Clinton Gormley) #2

Yes, you can use version_type=external and version=$your_version in
index/delete operations. See
http://www.elasticsearch.org/guide/reference/api/index_/

From 1.00 you'll also be able to specify version numbers for update
actions. See

clint

On 6 September 2013 11:17, Amit Soni amitsoni29@gmail.com wrote:

Hi everyone - I am wondering if ElasticSearch provides a mechanism wherein
an insert/update of a document is allowed only if a specified condition is
met.
Lets say that I have a document with id=id1 and my_version=8. Now an
update to this document should be allowed only if the field my_version has
a value greater than 8. If not, the update should not be done.

For my use case, I have more than one applications (message bus) entering
data into ES server and I want to ensure that an existing copy of the
document does not get overwritten with a stale copy, hence I have a
versioning system of my own.

Does ElasticSearch provide a way to do this?

-Amit.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Amit Soni) #3

Hi Clint - This is super helpful, thanks a lot! A follow up question: Do I
need to specify version_type=external with each index request or can I
specify it one time in setting/mapping while creating the index? I tried
to search on Google for examples but couldn't find any.

-Amit.

On Fri, Sep 6, 2013 at 3:30 AM, Clinton Gormley clint@traveljury.comwrote:

Yes, you can use version_type=external and version=$your_version in
index/delete operations. See
http://www.elasticsearch.org/guide/reference/api/index_/

From 1.00 you'll also be able to specify version numbers for update
actions. See
https://github.com/elasticsearch/elasticsearch/issues/3111?source=cc

clint

On 6 September 2013 11:17, Amit Soni amitsoni29@gmail.com wrote:

Hi everyone - I am wondering if ElasticSearch provides a mechanism
wherein an insert/update of a document is allowed only if a specified
condition is met.
Lets say that I have a document with id=id1 and my_version=8. Now an
update to this document should be allowed only if the field my_version has
a value greater than 8. If not, the update should not be done.

For my use case, I have more than one applications (message bus) entering
data into ES server and I want to ensure that an existing copy of the
document does not get overwritten with a stale copy, hence I have a
versioning system of my own.

Does ElasticSearch provide a way to do this?

-Amit.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Brian Yoder) #4

Amit,

Here is a version of the documentation I wrote after reading the on-line
documentation and then experimenting:

The version number is internal or external on a per-update request.
ElasticSearch implements the following behaviors:

  1. Internal version numbers for a document start at 1 and increment by one
    after each update to a document. When indexing or deleting an existing
    document, an internal version number must match the document's version or
    the operation will be rejected.

If the version number is specified as internal, then: If the document
doesn't exist, it must start at 1, and if the document exists its version
must be equal to the specified version number.

  1. External version numbers for a document start at whatever value is
    specified by the application. When indexing or deleting an existing
    document, an external version number must be greater than the document's
    version or the operation will be rejected. When the operation completes
    successfully, the document's version will be the externally specified
    version number.

If the version number is specified as external, then: The document will be
updated as assigned this version if it doesn't already exist, or if it
exists and the specified version is greater than the document's existing
version.

Brian

On Friday, September 6, 2013 5:18:09 PM UTC-4, amit.soni wrote:

Hi Clint - This is super helpful, thanks a lot! A follow up question: Do I
need to specify version_type=external with each index request or can I
specify it one time in setting/mapping while creating the index? I tried
to search on Google for examples but couldn't find any.

-Amit.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Amit Soni) #5

Thanks much Brian, appreciate your response!

-Amit.

On Fri, Sep 6, 2013 at 4:33 PM, InquiringMind brian.from.fl@gmail.comwrote:

Amit,

Here is a version of the documentation I wrote after reading the on-line
documentation and then experimenting:

The version number is internal or external on a per-update request.
ElasticSearch implements the following behaviors:

  1. Internal version numbers for a document start at 1 and increment by one
    after each update to a document. When indexing or deleting an existing
    document, an internal version number must match the document's version or
    the operation will be rejected.

If the version number is specified as internal, then: If the document
doesn't exist, it must start at 1, and if the document exists its version
must be equal to the specified version number.

  1. External version numbers for a document start at whatever value is
    specified by the application. When indexing or deleting an existing
    document, an external version number must be greater than the document's
    version or the operation will be rejected. When the operation completes
    successfully, the document's version will be the externally specified
    version number.

If the version number is specified as external, then: The document will be
updated as assigned this version if it doesn't already exist, or if it
exists and the specified version is greater than the document's existing
version.

Brian

On Friday, September 6, 2013 5:18:09 PM UTC-4, amit.soni wrote:

Hi Clint - This is super helpful, thanks a lot! A follow up question: Do
I need to specify version_type=external with each index request or can I
specify it one time in setting/mapping while creating the index? I tried
to search on Google for examples but couldn't find any.

-Amit.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Amit Soni) #6

Actually I have a follow up question: I am using Java APIs and I haven't
found a way to specify query params in the url (...&version_type=external).

Any recommendations on how to specify the version_type when using Java API?

-Amit.

On Sat, Sep 7, 2013 at 2:32 PM, Amit Soni amitsoni29@gmail.com wrote:

Thanks much Brian, appreciate your response!

-Amit.

On Fri, Sep 6, 2013 at 4:33 PM, InquiringMind brian.from.fl@gmail.comwrote:

Amit,

Here is a version of the documentation I wrote after reading the on-line
documentation and then experimenting:

The version number is internal or external on a per-update request.
ElasticSearch implements the following behaviors:

  1. Internal version numbers for a document start at 1 and increment by
    one after each update to a document. When indexing or deleting an existing
    document, an internal version number must match the document's version or
    the operation will be rejected.

If the version number is specified as internal, then: If the document
doesn't exist, it must start at 1, and if the document exists its version
must be equal to the specified version number.

  1. External version numbers for a document start at whatever value is
    specified by the application. When indexing or deleting an existing
    document, an external version number must be greater than the document's
    version or the operation will be rejected. When the operation completes
    successfully, the document's version will be the externally specified
    version number.

If the version number is specified as external, then: The document will
be updated as assigned this version if it doesn't already exist, or if it
exists and the specified version is greater than the document's existing
version.

Brian

On Friday, September 6, 2013 5:18:09 PM UTC-4, amit.soni wrote:

Hi Clint - This is super helpful, thanks a lot! A follow up question: Do
I need to specify version_type=external with each index request or can I
specify it one time in setting/mapping while creating the index? I tried
to search on Google for examples but couldn't find any.

-Amit.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Luca Cavanna) #7

If you use the Java API you don't use the REST endpoint but directly the
binary format that elasticsearch uses internally for inter-node
communications. That's the very same format all REST requests are
translated to. Every api that supports the version_type parameter exposes
proper setters for it. It is called setVersionType on the request builder
and versionType on the Request class.

Cheers
Luca

On Wednesday, September 11, 2013 2:50:02 AM UTC+2, amit.soni wrote:

Actually I have a follow up question: I am using Java APIs and I haven't
found a way to specify query params in the url (...&version_type=external)
.

Any recommendations on how to specify the version_type when using Java API?

-Amit.

On Sat, Sep 7, 2013 at 2:32 PM, Amit Soni <amits...@gmail.com<javascript:>

wrote:

Thanks much Brian, appreciate your response!

-Amit.

On Fri, Sep 6, 2013 at 4:33 PM, InquiringMind <brian....@gmail.com<javascript:>

wrote:

Amit,

Here is a version of the documentation I wrote after reading the on-line
documentation and then experimenting:

The version number is internal or external on a per-update request.
ElasticSearch implements the following behaviors:

  1. Internal version numbers for a document start at 1 and increment by
    one after each update to a document. When indexing or deleting an existing
    document, an internal version number must match the document's version or
    the operation will be rejected.

If the version number is specified as internal, then: If the document
doesn't exist, it must start at 1, and if the document exists its version
must be equal to the specified version number.

  1. External version numbers for a document start at whatever value is
    specified by the application. When indexing or deleting an existing
    document, an external version number must be greater than the document's
    version or the operation will be rejected. When the operation completes
    successfully, the document's version will be the externally specified
    version number.

If the version number is specified as external, then: The document will
be updated as assigned this version if it doesn't already exist, or if it
exists and the specified version is greater than the document's existing
version.

Brian

On Friday, September 6, 2013 5:18:09 PM UTC-4, amit.soni wrote:

Hi Clint - This is super helpful, thanks a lot! A follow up question:
Do I need to specify version_type=external with each index request or can I
specify it one time in setting/mapping while creating the index? I tried
to search on Google for examples but couldn't find any.

-Amit.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Brian Yoder) #8

Amit,

Something along the lines of the following.

Note that the "rec" variable is my own object (record == document) and
the methods should be fairly self-evident: rec.hasVersion returns true
if a version is specified; rec.isVersionExternal returns true for an
external version and false for an internal version, and its other
methods should be self-evident as well:

import org.elasticsearch.action.index.*;
import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.common.unit.TimeValue;
import org.elasticsearch.index.VersionType;

IndexRequestBuilder irb = client.prepareIndex(rec.getIndex(),
rec.getType(), rec.getId());

irb.setSource(rec.sourceToJson()).setOpType(IndexRequest.OpType.INDEX);
irb.setRefresh(refresh);

if (rec.hasVersion())
{
irb.setVersion(rec.getVersion());
if (rec.isVersionExternal())
irb.setVersionType(VersionType.EXTERNAL);
}

Hope this helps!

Brian

On Tuesday, September 10, 2013 8:50:02 PM UTC-4, amit.soni wrote:

Actually I have a follow up question: I am using Java APIs and I haven't
found a way to specify query params in the url (...&version_type=external)
.

Any recommendations on how to specify the version_type when using Java API?

-Amit.

On Sat, Sep 7, 2013 at 2:32 PM, Amit Soni <amits...@gmail.com<javascript:>

wrote:

Thanks much Brian, appreciate your response!

-Amit.

On Fri, Sep 6, 2013 at 4:33 PM, InquiringMind <brian....@gmail.com<javascript:>

wrote:

Amit,

Here is a version of the documentation I wrote after reading the on-line
documentation and then experimenting:

The version number is internal or external on a per-update request.
ElasticSearch implements the following behaviors:

  1. Internal version numbers for a document start at 1 and increment by
    one after each update to a document. When indexing or deleting an existing
    document, an internal version number must match the document's version or
    the operation will be rejected.

If the version number is specified as internal, then: If the document
doesn't exist, it must start at 1, and if the document exists its version
must be equal to the specified version number.

  1. External version numbers for a document start at whatever value is
    specified by the application. When indexing or deleting an existing
    document, an external version number must be greater than the document's
    version or the operation will be rejected. When the operation completes
    successfully, the document's version will be the externally specified
    version number.

If the version number is specified as external, then: The document will
be updated as assigned this version if it doesn't already exist, or if it
exists and the specified version is greater than the document's existing
version.

Brian

On Friday, September 6, 2013 5:18:09 PM UTC-4, amit.soni wrote:

Hi Clint - This is super helpful, thanks a lot! A follow up question:
Do I need to specify version_type=external with each index request or can I
specify it one time in setting/mapping while creating the index? I tried
to search on Google for examples but couldn't find any.

-Amit.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Amit Soni) #9

Hi Luca - My bad, I should have additionally specified that I have been
using JEST client to make REST based calls. This puts a limitation in me
specifying verstion_type in the request url.

-Amit.

On Wed, Sep 11, 2013 at 5:07 AM, Luca Cavanna cavannaluca@gmail.com wrote:

If you use the Java API you don't use the REST endpoint but directly the
binary format that elasticsearch uses internally for inter-node
communications. That's the very same format all REST requests are
translated to. Every api that supports the version_type parameter exposes
proper setters for it. It is called setVersionType on the request builder
and versionType on the Request class.

Cheers
Luca

On Wednesday, September 11, 2013 2:50:02 AM UTC+2, amit.soni wrote:

Actually I have a follow up question: I am using Java APIs and I haven't
found a way to specify query params in the url (...&version_type=
external).

Any recommendations on how to specify the version_type when using Java
API?

-Amit.

On Sat, Sep 7, 2013 at 2:32 PM, Amit Soni amits...@gmail.com wrote:

Thanks much Brian, appreciate your response!

-Amit.

On Fri, Sep 6, 2013 at 4:33 PM, InquiringMind brian....@gmail.comwrote:

Amit,

Here is a version of the documentation I wrote after reading the
on-line documentation and then experimenting:

The version number is internal or external on a per-update request.
ElasticSearch implements the following behaviors:

  1. Internal version numbers for a document start at 1 and increment by
    one after each update to a document. When indexing or deleting an existing
    document, an internal version number must match the document's version or
    the operation will be rejected.

If the version number is specified as internal, then: If the document
doesn't exist, it must start at 1, and if the document exists its version
must be equal to the specified version number.

  1. External version numbers for a document start at whatever value is
    specified by the application. When indexing or deleting an existing
    document, an external version number must be greater than the document's
    version or the operation will be rejected. When the operation completes
    successfully, the document's version will be the externally specified
    version number.

If the version number is specified as external, then: The document will
be updated as assigned this version if it doesn't already exist, or if it
exists and the specified version is greater than the document's existing
version.

Brian

On Friday, September 6, 2013 5:18:09 PM UTC-4, amit.soni wrote:

Hi Clint - This is super helpful, thanks a lot! A follow up question:
Do I need to specify version_type=external with each index request or can I
specify it one time in setting/mapping while creating the index? I tried
to search on Google for examples but couldn't find any.

-Amit.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@**googlegroups.com.

For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out
.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Luca Cavanna) #10

I see! I'm not familiar enough with Jest to answer this question then. I
had a brief look at it but couldn't find the answer, sorry about that.

On Wednesday, September 11, 2013 10:30:25 PM UTC+2, amit.soni wrote:

Hi Luca - My bad, I should have additionally specified that I have been
using JEST client to make REST based calls. This puts a limitation in me
specifying verstion_type in the request url.

-Amit.

On Wed, Sep 11, 2013 at 5:07 AM, Luca Cavanna <cavan...@gmail.com<javascript:>

wrote:

If you use the Java API you don't use the REST endpoint but directly the
binary format that elasticsearch uses internally for inter-node
communications. That's the very same format all REST requests are
translated to. Every api that supports the version_type parameter exposes
proper setters for it. It is called setVersionType on the request builder
and versionType on the Request class.

Cheers
Luca

On Wednesday, September 11, 2013 2:50:02 AM UTC+2, amit.soni wrote:

Actually I have a follow up question: I am using Java APIs and I haven't
found a way to specify query params in the url (...&version_type=
external).

Any recommendations on how to specify the version_type when using Java
API?

-Amit.

On Sat, Sep 7, 2013 at 2:32 PM, Amit Soni amits...@gmail.com wrote:

Thanks much Brian, appreciate your response!

-Amit.

On Fri, Sep 6, 2013 at 4:33 PM, InquiringMind brian....@gmail.comwrote:

Amit,

Here is a version of the documentation I wrote after reading the
on-line documentation and then experimenting:

The version number is internal or external on a per-update request.
ElasticSearch implements the following behaviors:

  1. Internal version numbers for a document start at 1 and increment by
    one after each update to a document. When indexing or deleting an existing
    document, an internal version number must match the document's version or
    the operation will be rejected.

If the version number is specified as internal, then: If the document
doesn't exist, it must start at 1, and if the document exists its version
must be equal to the specified version number.

  1. External version numbers for a document start at whatever value is
    specified by the application. When indexing or deleting an existing
    document, an external version number must be greater than the document's
    version or the operation will be rejected. When the operation completes
    successfully, the document's version will be the externally specified
    version number.

If the version number is specified as external, then: The document
will be updated as assigned this version if it doesn't already exist, or if
it exists and the specified version is greater than the document's existing
version.

Brian

On Friday, September 6, 2013 5:18:09 PM UTC-4, amit.soni wrote:

Hi Clint - This is super helpful, thanks a lot! A follow up question:
Do I need to specify version_type=external with each index request or can I
specify it one time in setting/mapping while creating the index? I tried
to search on Google for examples but couldn't find any.

-Amit.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@**googlegroups.com.

For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out
.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Amit Soni) #11

Thanks Luca. Do you have a recommendation on what approach to take in order
to make REST based calls from Java application (apart from using JEST).

-Amit.

On Thu, Sep 12, 2013 at 4:27 AM, Luca Cavanna cavannaluca@gmail.com wrote:

I see! I'm not familiar enough with Jest to answer this question then. I
had a brief look at it but couldn't find the answer, sorry about that.

On Wednesday, September 11, 2013 10:30:25 PM UTC+2, amit.soni wrote:

Hi Luca - My bad, I should have additionally specified that I have been
using JEST client to make REST based calls. This puts a limitation in me
specifying verstion_type in the request url.

-Amit.

On Wed, Sep 11, 2013 at 5:07 AM, Luca Cavanna cavan...@gmail.com wrote:

If you use the Java API you don't use the REST endpoint but directly the
binary format that elasticsearch uses internally for inter-node
communications. That's the very same format all REST requests are
translated to. Every api that supports the version_type parameter exposes
proper setters for it. It is called setVersionType on the request builder
and versionType on the Request class.

Cheers
Luca

On Wednesday, September 11, 2013 2:50:02 AM UTC+2, amit.soni wrote:

Actually I have a follow up question: I am using Java APIs and I
haven't found a way to specify query params in the url (...&version_type=
external).

Any recommendations on how to specify the version_type when using Java
API?

-Amit.

On Sat, Sep 7, 2013 at 2:32 PM, Amit Soni amits...@gmail.com wrote:

Thanks much Brian, appreciate your response!

-Amit.

On Fri, Sep 6, 2013 at 4:33 PM, InquiringMind brian....@gmail.comwrote:

Amit,

Here is a version of the documentation I wrote after reading the
on-line documentation and then experimenting:

The version number is internal or external on a per-update request.
ElasticSearch implements the following behaviors:

  1. Internal version numbers for a document start at 1 and increment
    by one after each update to a document. When indexing or deleting an
    existing document, an internal version number must match the document's
    version or the operation will be rejected.

If the version number is specified as internal, then: If the document
doesn't exist, it must start at 1, and if the document exists its version
must be equal to the specified version number.

  1. External version numbers for a document start at whatever value is
    specified by the application. When indexing or deleting an existing
    document, an external version number must be greater than the document's
    version or the operation will be rejected. When the operation completes
    successfully, the document's version will be the externally specified
    version number.

If the version number is specified as external, then: The document
will be updated as assigned this version if it doesn't already exist, or if
it exists and the specified version is greater than the document's existing
version.

Brian

On Friday, September 6, 2013 5:18:09 PM UTC-4, amit.soni wrote:

Hi Clint - This is super helpful, thanks a lot! A follow up
question: Do I need to specify version_type=external with each index
request or can I specify it one time in setting/mapping while creating the
index? I tried to search on Google for examples but couldn't find any.

-Amit.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@**googlegroups.**com.

For more options, visit https://groups.google.com/groups/opt_outhttps://groups.google.com/groups/opt_out
.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@**googlegroups.com.
For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out
.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Luca Cavanna) #12

Not really, we don't currently have a real client written in Java that uses
the REST apis. JEST is the only one I'm aware of. On the other hand it is
quite easy to send http requests through whatever http client.

May I ask what is the reason why you chose not to use for instance the
TransportClient provided with the Java API?

On Thu, Sep 12, 2013 at 11:13 PM, Amit Soni amitsoni29@gmail.com wrote:

Thanks Luca. Do you have a recommendation on what approach to take in
order to make REST based calls from Java application (apart from using
JEST).

-Amit.

On Thu, Sep 12, 2013 at 4:27 AM, Luca Cavanna cavannaluca@gmail.comwrote:

I see! I'm not familiar enough with Jest to answer this question then. I
had a brief look at it but couldn't find the answer, sorry about that.

On Wednesday, September 11, 2013 10:30:25 PM UTC+2, amit.soni wrote:

Hi Luca - My bad, I should have additionally specified that I have been
using JEST client to make REST based calls. This puts a limitation in me
specifying verstion_type in the request url.

-Amit.

On Wed, Sep 11, 2013 at 5:07 AM, Luca Cavanna cavan...@gmail.comwrote:

If you use the Java API you don't use the REST endpoint but directly
the binary format that elasticsearch uses internally for inter-node
communications. That's the very same format all REST requests are
translated to. Every api that supports the version_type parameter exposes
proper setters for it. It is called setVersionType on the request builder
and versionType on the Request class.

Cheers
Luca

On Wednesday, September 11, 2013 2:50:02 AM UTC+2, amit.soni wrote:

Actually I have a follow up question: I am using Java APIs and I
haven't found a way to specify query params in the url (...&version_type=
external).

Any recommendations on how to specify the version_type when using Java
API?

-Amit.

On Sat, Sep 7, 2013 at 2:32 PM, Amit Soni amits...@gmail.com wrote:

Thanks much Brian, appreciate your response!

-Amit.

On Fri, Sep 6, 2013 at 4:33 PM, InquiringMind brian....@gmail.comwrote:

Amit,

Here is a version of the documentation I wrote after reading the
on-line documentation and then experimenting:

The version number is internal or external on a per-update request.
ElasticSearch implements the following behaviors:

  1. Internal version numbers for a document start at 1 and increment
    by one after each update to a document. When indexing or deleting an
    existing document, an internal version number must match the document's
    version or the operation will be rejected.

If the version number is specified as internal, then: If the
document doesn't exist, it must start at 1, and if the document exists its
version must be equal to the specified version number.

  1. External version numbers for a document start at whatever value
    is specified by the application. When indexing or deleting an existing
    document, an external version number must be greater than the document's
    version or the operation will be rejected. When the operation completes
    successfully, the document's version will be the externally specified
    version number.

If the version number is specified as external, then: The document
will be updated as assigned this version if it doesn't already exist, or if
it exists and the specified version is greater than the document's existing
version.

Brian

On Friday, September 6, 2013 5:18:09 PM UTC-4, amit.soni wrote:

Hi Clint - This is super helpful, thanks a lot! A follow up
question: Do I need to specify version_type=external with each index
request or can I specify it one time in setting/mapping while creating the
index? I tried to search on Google for examples but couldn't find any.

-Amit.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@**googlegroups.**com.

For more options, visit https://groups.google.com/groups/opt_outhttps://groups.google.com/groups/opt_out
.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@**googlegroups.com.
For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out
.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/nRUSChH-1h8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Amit Soni) #13

Luca - That is a good question. Reading through some of the articles on the
web, I realized that it is much easier to debug problems if the
communication happens using http/json. I can log the request and play it
easily.
Secondly, using TransportClient seems to be including client code as part
of the cluster and I think client should be outside of the cluster.

I would love to hear your thoughts on the same. Since we are setting up the
infrastructure at this point, it would help to know the pros and cons.

-Amit.

On Thu, Sep 12, 2013 at 2:32 PM, Luca Cavanna cavannaluca@gmail.com wrote:

Not really, we don't currently have a real client written in Java that
uses the REST apis. JEST is the only one I'm aware of. On the other hand it
is quite easy to send http requests through whatever http client.

May I ask what is the reason why you chose not to use for instance the
TransportClient provided with the Java API?

On Thu, Sep 12, 2013 at 11:13 PM, Amit Soni amitsoni29@gmail.com wrote:

Thanks Luca. Do you have a recommendation on what approach to take in
order to make REST based calls from Java application (apart from using
JEST).

-Amit.

On Thu, Sep 12, 2013 at 4:27 AM, Luca Cavanna cavannaluca@gmail.comwrote:

I see! I'm not familiar enough with Jest to answer this question then. I
had a brief look at it but couldn't find the answer, sorry about that.

On Wednesday, September 11, 2013 10:30:25 PM UTC+2, amit.soni wrote:

Hi Luca - My bad, I should have additionally specified that I have been
using JEST client to make REST based calls. This puts a limitation in me
specifying verstion_type in the request url.

-Amit.

On Wed, Sep 11, 2013 at 5:07 AM, Luca Cavanna cavan...@gmail.comwrote:

If you use the Java API you don't use the REST endpoint but directly
the binary format that elasticsearch uses internally for inter-node
communications. That's the very same format all REST requests are
translated to. Every api that supports the version_type parameter exposes
proper setters for it. It is called setVersionType on the request builder
and versionType on the Request class.

Cheers
Luca

On Wednesday, September 11, 2013 2:50:02 AM UTC+2, amit.soni wrote:

Actually I have a follow up question: I am using Java APIs and I
haven't found a way to specify query params in the url (...&version_type=
external).

Any recommendations on how to specify the version_type when using
Java API?

-Amit.

On Sat, Sep 7, 2013 at 2:32 PM, Amit Soni amits...@gmail.com wrote:

Thanks much Brian, appreciate your response!

-Amit.

On Fri, Sep 6, 2013 at 4:33 PM, InquiringMind brian....@gmail.comwrote:

Amit,

Here is a version of the documentation I wrote after reading the
on-line documentation and then experimenting:

The version number is internal or external on a per-update request.
ElasticSearch implements the following behaviors:

  1. Internal version numbers for a document start at 1 and increment
    by one after each update to a document. When indexing or deleting an
    existing document, an internal version number must match the document's
    version or the operation will be rejected.

If the version number is specified as internal, then: If the
document doesn't exist, it must start at 1, and if the document exists its
version must be equal to the specified version number.

  1. External version numbers for a document start at whatever value
    is specified by the application. When indexing or deleting an existing
    document, an external version number must be greater than the document's
    version or the operation will be rejected. When the operation completes
    successfully, the document's version will be the externally specified
    version number.

If the version number is specified as external, then: The document
will be updated as assigned this version if it doesn't already exist, or if
it exists and the specified version is greater than the document's existing
version.

Brian

On Friday, September 6, 2013 5:18:09 PM UTC-4, amit.soni wrote:

Hi Clint - This is super helpful, thanks a lot! A follow up
question: Do I need to specify version_type=external with each index
request or can I specify it one time in setting/mapping while creating the
index? I tried to search on Google for examples but couldn't find any.

-Amit.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@**googlegroups.**com.

For more options, visit https://groups.google.com/grou
ps/opt_out https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@**googlegroups.com.
For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out
.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/nRUSChH-1h8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Luca Cavanna) #14

Thanks for your insight.

By the way if you use the TransportClient you effectively don't join the
cluster, but connect to some nodes in round-robin fashion, which seems to
be what you're looking for. I'd give it a try if you haven't done it yet.

I think we might provide a Java REST client in the future to address the
facet that the Java API is powerful, but sometimes too much for a client.
On the other hand the performance should be better and it is not that
different from using rest, I mean to debug as you mentioned it as the first
point.

Cheers
Luca

On Fri, Sep 13, 2013 at 7:36 PM, Amit Soni amitsoni29@gmail.com wrote:

Luca - That is a good question. Reading through some of the articles on
the web, I realized that it is much easier to debug problems if the
communication happens using http/json. I can log the request and play it
easily.
Secondly, using TransportClient seems to be including client code as part
of the cluster and I think client should be outside of the cluster.

I would love to hear your thoughts on the same. Since we are setting up
the infrastructure at this point, it would help to know the pros and cons.

-Amit.

On Thu, Sep 12, 2013 at 2:32 PM, Luca Cavanna cavannaluca@gmail.comwrote:

Not really, we don't currently have a real client written in Java that
uses the REST apis. JEST is the only one I'm aware of. On the other hand it
is quite easy to send http requests through whatever http client.

May I ask what is the reason why you chose not to use for instance the
TransportClient provided with the Java API?

On Thu, Sep 12, 2013 at 11:13 PM, Amit Soni amitsoni29@gmail.com wrote:

Thanks Luca. Do you have a recommendation on what approach to take in
order to make REST based calls from Java application (apart from using
JEST).

-Amit.

On Thu, Sep 12, 2013 at 4:27 AM, Luca Cavanna cavannaluca@gmail.comwrote:

I see! I'm not familiar enough with Jest to answer this question then.
I had a brief look at it but couldn't find the answer, sorry about that.

On Wednesday, September 11, 2013 10:30:25 PM UTC+2, amit.soni wrote:

Hi Luca - My bad, I should have additionally specified that I have
been using JEST client to make REST based calls. This puts a limitation in
me specifying verstion_type in the request url.

-Amit.

On Wed, Sep 11, 2013 at 5:07 AM, Luca Cavanna cavan...@gmail.comwrote:

If you use the Java API you don't use the REST endpoint but directly
the binary format that elasticsearch uses internally for inter-node
communications. That's the very same format all REST requests are
translated to. Every api that supports the version_type parameter exposes
proper setters for it. It is called setVersionType on the request builder
and versionType on the Request class.

Cheers
Luca

On Wednesday, September 11, 2013 2:50:02 AM UTC+2, amit.soni wrote:

Actually I have a follow up question: I am using Java APIs and I
haven't found a way to specify query params in the url (...&version_type=
external).

Any recommendations on how to specify the version_type when using
Java API?

-Amit.

On Sat, Sep 7, 2013 at 2:32 PM, Amit Soni amits...@gmail.comwrote:

Thanks much Brian, appreciate your response!

-Amit.

On Fri, Sep 6, 2013 at 4:33 PM, InquiringMind brian....@gmail.comwrote:

Amit,

Here is a version of the documentation I wrote after reading the
on-line documentation and then experimenting:

The version number is internal or external on a per-update
request. ElasticSearch implements the following behaviors:

  1. Internal version numbers for a document start at 1 and
    increment by one after each update to a document. When indexing or deleting
    an existing document, an internal version number must match the document's
    version or the operation will be rejected.

If the version number is specified as internal, then: If the
document doesn't exist, it must start at 1, and if the document exists its
version must be equal to the specified version number.

  1. External version numbers for a document start at whatever value
    is specified by the application. When indexing or deleting an existing
    document, an external version number must be greater than the document's
    version or the operation will be rejected. When the operation completes
    successfully, the document's version will be the externally specified
    version number.

If the version number is specified as external, then: The document
will be updated as assigned this version if it doesn't already exist, or if
it exists and the specified version is greater than the document's existing
version.

Brian

On Friday, September 6, 2013 5:18:09 PM UTC-4, amit.soni wrote:

Hi Clint - This is super helpful, thanks a lot! A follow up
question: Do I need to specify version_type=external with each index
request or can I specify it one time in setting/mapping while creating the
index? I tried to search on Google for examples but couldn't find any.

-Amit.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@**googlegroups.**com.

For more options, visit https://groups.google.com/grou
ps/opt_out https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@**googlegroups.com.
For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out
.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/nRUSChH-1h8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/nRUSChH-1h8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Simpsora) #15

Amit,

I'm using the Java API and a TransportClient. In order to debug queries, I
use something like the following:

private String getSearchURL( SearchRequestBuilder builder, String indexName,
String typeName ) {
String jsonQuery = builder.internalBuilder.toXContent( JsonXContent.
contentBuilder(), ToXContent.EMPTY_PARAMS ).prettyPrint().string();
StringBuilder sb = new StringBuilder( "http://" )
.append( configService.getValue( "search.server" ) )
.append( ":" )
.append( configService.getValue( "search.http.query.port" ) )
.append( "/" )
.append( indexName )
.append( "/" )
.append( typeName )
.append( "/_search" )
.append( "?pretty" )
.append( "&source=" )
.append( URLEncoder.encode( jsonQuery ) );
return sb.toString();
}

The source URL parameter allows you to pass in a pre-constructed query in
Json format in a GET request.

This returns an HTTP GET URL you can stick in a browser and debug queries
all you want.

Ross

On Saturday, 14 September 2013 03:36:21 UTC+10, amit.soni wrote:

Luca - That is a good question. Reading through some of the articles on
the web, I realized that it is much easier to debug problems if the
communication happens using http/json. I can log the request and play it
easily.
Secondly, using TransportClient seems to be including client code as part
of the cluster and I think client should be outside of the cluster.

I would love to hear your thoughts on the same. Since we are setting up
the infrastructure at this point, it would help to know the pros and cons.

-Amit.

On Thu, Sep 12, 2013 at 2:32 PM, Luca Cavanna <cavan...@gmail.com<javascript:>

wrote:

Not really, we don't currently have a real client written in Java that
uses the REST apis. JEST is the only one I'm aware of. On the other hand it
is quite easy to send http requests through whatever http client.

May I ask what is the reason why you chose not to use for instance the
TransportClient provided with the Java API?

On Thu, Sep 12, 2013 at 11:13 PM, Amit Soni <amits...@gmail.com<javascript:>

wrote:

Thanks Luca. Do you have a recommendation on what approach to take in
order to make REST based calls from Java application (apart from using
JEST).

-Amit.

On Thu, Sep 12, 2013 at 4:27 AM, Luca Cavanna <cavan...@gmail.com<javascript:>

wrote:

I see! I'm not familiar enough with Jest to answer this question then.
I had a brief look at it but couldn't find the answer, sorry about that.

On Wednesday, September 11, 2013 10:30:25 PM UTC+2, amit.soni wrote:

Hi Luca - My bad, I should have additionally specified that I have
been using JEST client to make REST based calls. This puts a limitation in
me specifying verstion_type in the request url.

-Amit.

On Wed, Sep 11, 2013 at 5:07 AM, Luca Cavanna cavan...@gmail.comwrote:

If you use the Java API you don't use the REST endpoint but directly
the binary format that elasticsearch uses internally for inter-node
communications. That's the very same format all REST requests are
translated to. Every api that supports the version_type parameter exposes
proper setters for it. It is called setVersionType on the request builder
and versionType on the Request class.

Cheers
Luca

On Wednesday, September 11, 2013 2:50:02 AM UTC+2, amit.soni wrote:

Actually I have a follow up question: I am using Java APIs and I
haven't found a way to specify query params in the url (...&version_type=
external).

Any recommendations on how to specify the version_type when using
Java API?

-Amit.

On Sat, Sep 7, 2013 at 2:32 PM, Amit Soni amits...@gmail.comwrote:

Thanks much Brian, appreciate your response!

-Amit.

On Fri, Sep 6, 2013 at 4:33 PM, InquiringMind brian....@gmail.comwrote:

Amit,

Here is a version of the documentation I wrote after reading the
on-line documentation and then experimenting:

The version number is internal or external on a per-update
request. ElasticSearch implements the following behaviors:

  1. Internal version numbers for a document start at 1 and
    increment by one after each update to a document. When indexing or deleting
    an existing document, an internal version number must match the document's
    version or the operation will be rejected.

If the version number is specified as internal, then: If the
document doesn't exist, it must start at 1, and if the document exists its
version must be equal to the specified version number.

  1. External version numbers for a document start at whatever value
    is specified by the application. When indexing or deleting an existing
    document, an external version number must be greater than the document's
    version or the operation will be rejected. When the operation completes
    successfully, the document's version will be the externally specified
    version number.

If the version number is specified as external, then: The document
will be updated as assigned this version if it doesn't already exist, or if
it exists and the specified version is greater than the document's existing
version.

Brian

On Friday, September 6, 2013 5:18:09 PM UTC-4, amit.soni wrote:

Hi Clint - This is super helpful, thanks a lot! A follow up
question: Do I need to specify version_type=external with each index
request or can I specify it one time in setting/mapping while creating the
index? I tried to search on Google for examples but couldn't find any.

-Amit.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@**googlegroups.**com.

For more options, visit https://groups.google.com/grou
ps/opt_out https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@**googlegroups.com.
For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out
.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/nRUSChH-1h8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com <javascript:>.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #16