Java API synchronous vs asynchronous?

echin1999_2 · April 13, 2014, 3:09am

Hi.

I'm adding to my elasticSearch database (v. 0.90) an object via this java
call.

IndexResponse response = _client.prepareIndex(SomeIndex, SomeType,
SomeId).setSource(SomeJSON).execute().actionGet();

If I do a search for this object immediately after, i notice i sometimes
don't find it. I assume this is because it is an asynchronous call. Is
that correct?
The documentation mentions the following:

The index API allows to set the threading model the operation will be
performed when the actual execution of the API is performed on the same
node (the API is executed on a shard that is allocated on the same server).

The options are to execute the operation on a different thread, or to
execute it on the calling thread (note that the API is still async). By
default, operationThreaded is set totrue which means the operation is
executed on a different thread.

Will setting operationThreaded to false guarantee that the object be
available for search immediately after? The snippet above mentions "(note
that the API is still async)" - so I'm guessing no. Is there a way to
make the api call synchronous so i can be assured that on my next search
for the object, it will definitely be there?

thanks
Ed

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/766927ea-6943-44f5-8da2-997d1ff1b2f6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

dadoonet · April 13, 2014, 5:47am

It won't be searchable until the next refresh.
GET is real time.
SEARCH is near real time.

That could explain here.

--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 13 avr. 2014 à 05:09, echin1999 echin1999@gmail.com a écrit :

Hi.

I'm adding to my elasticSearch database (v. 0.90) an object via this java call.

IndexResponse response = _client.prepareIndex(SomeIndex, SomeType, SomeId).setSource(SomeJSON).execute().actionGet();

If I do a search for this object immediately after, i notice i sometimes don't find it. I assume this is because it is an asynchronous call. Is that correct?

The documentation mentions the following:

The index API allows to set the threading model the operation will be performed when the actual execution of the API is performed on the same node (the API is executed on a shard that is allocated on the same server).

The options are to execute the operation on a different thread, or to execute it on the calling thread (note that the API is still async). By default, operationThreaded is set totrue which means the operation is executed on a different thread.

Will setting operationThreaded to false guarantee that the object be available for search immediately after? The snippet above mentions "(note that the API is still async)" - so I'm guessing no. Is there a way to make the api call synchronous so i can be assured that on my next search for the object, it will definitely be there?

thanks
Ed

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/766927ea-6943-44f5-8da2-997d1ff1b2f6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CFA2AD64-8C61-44E7-A63A-79B02B257EE5%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

echin1999_2 · April 13, 2014, 1:56pm

I see. From looking at the documentation for "refresh" it looks like the default interval is 1 second? so yes, that would explain my scenario. I wasn't aware of the "refresh".

do you know if the interval can be decreased to less than a second? I understand there is impact to the performance of indexing if this value is lowered. Is there also impact to searching with respect to this interval?

thanks!
Ed

On Apr 13, 2014, at 1:47 AM, David Pilato wrote:

It won't be searchable until the next refresh.
GET is real time.
SEARCH is near real time.

That could explain here.

--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 13 avr. 2014 à 05:09, echin1999 echin1999@gmail.com a écrit :

Hi.

I'm adding to my elasticSearch database (v. 0.90) an object via this java call.

IndexResponse response = _client.prepareIndex(SomeIndex, SomeType, SomeId).setSource(SomeJSON).execute().actionGet();

If I do a search for this object immediately after, i notice i sometimes don't find it. I assume this is because it is an asynchronous call. Is that correct?

The documentation mentions the following:

The index API allows to set the threading model the operation will be performed when the actual execution of the API is performed on the same node (the API is executed on a shard that is allocated on the same server).

The options are to execute the operation on a different thread, or to execute it on the calling thread (note that the API is still async). By default, operationThreaded is set totrue which means the operation is executed on a different thread.

Will setting operationThreaded to false guarantee that the object be available for search immediately after? The snippet above mentions "(note that the API is still async)" - so I'm guessing no. Is there a way to make the api call synchronous so i can be assured that on my next search for the object, it will definitely be there?

thanks
Ed

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/766927ea-6943-44f5-8da2-997d1ff1b2f6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/DWN30UYzeN4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CFA2AD64-8C61-44E7-A63A-79B02B257EE5%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1A41EBDC-7539-4BAC-8C69-2B17B33ADCD0%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

jprante · April 13, 2014, 2:47pm

Look at IndexRequestBuilder, there is a method setRefresh(true) so you can
enforce a post refresh per index request.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHJay-pvs%3D4QoOyfYs%3DZkC_fQTtedO%2B6EeS0iyL_KHffA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

echin1999_2 · April 13, 2014, 2:54pm

If I do that, and there are concurrent threads doing the same thing, is there impact to threads that are simply searching? I don't mind the indexing threads being slow, but i don't want to impact the search threads too much.

thanks
ed

On Apr 13, 2014, at 10:47 AM, joergprante@gmail.com wrote:

Look at IndexRequestBuilder, there is a method setRefresh(true) so you can enforce a post refresh per index request.

Jörg

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/DWN30UYzeN4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHJay-pvs%3D4QoOyfYs%3DZkC_fQTtedO%2B6EeS0iyL_KHffA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5F90E219-EAF2-41AA-848D-99D27CB0952F%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

jprante · April 13, 2014, 3:34pm

There is massive impact - but try and see for yourself.

Jörg

On Sun, Apr 13, 2014 at 4:54 PM, Edward Chin echin1999@gmail.com wrote:

If I do that, and there are concurrent threads doing the same thing, is
there impact to threads that are simply searching? I don't mind the
indexing threads being slow, but i don't want to impact the search threads
too much.

thanks
ed

On Apr 13, 2014, at 10:47 AM, joergprante@gmail.com wrote:

Look at IndexRequestBuilder, there is a method setRefresh(true) so you can
enforce a post refresh per index request.

Jörg

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/DWN30UYzeN4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHJay-pvs%3D4QoOyfYs%3DZkC_fQTtedO%2B6EeS0iyL_KHffA%40mail.gmail.com https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHJay-pvs%3D4QoOyfYs%3DZkC_fQTtedO%2B6EeS0iyL_KHffA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5F90E219-EAF2-41AA-848D-99D27CB0952F%40gmail.com https://groups.google.com/d/msgid/elasticsearch/5F90E219-EAF2-41AA-848D-99D27CB0952F%40gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHeCXQX_NJfab4Wb00JFi8pPRf8GmpX8xnfuVym4yckMQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

mansishah · December 4, 2014, 7:38pm

I have a related question about synchronous behavior of Java APIs. I
understand refresh will make sure that the indexed document becomes
searchable but what about subsequent updates / deletes to the same document.

If I index a document and it is going to be done asynchronously does that
mean an immediate update or delete on this document could potentially fail
because the document is not yet actually saved? Note that the update/
delete could be coming from potentially different client but still
serially.

Is there a API like "flush" which guarantees that everything that has been
indexed so far is now available in ES (not for search but for
get/update/delete) at least in the trans log.

On Sunday, April 13, 2014 8:34:19 AM UTC-7, Jörg Prante wrote:

There is massive impact - but try and see for yourself.

Jörg

On Sun, Apr 13, 2014 at 4:54 PM, Edward Chin <echi...@gmail.com
<javascript:>> wrote:

If I do that, and there are concurrent threads doing the same thing, is
there impact to threads that are simply searching? I don't mind the
indexing threads being slow, but i don't want to impact the search threads
too much.

thanks
ed

On Apr 13, 2014, at 10:47 AM, joerg...@gmail.com <javascript:> wrote:

Look at IndexRequestBuilder, there is a method setRefresh(true) so you
can enforce a post refresh per index request.

Jörg

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/DWN30UYzeN4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHJay-pvs%3D4QoOyfYs%3DZkC_fQTtedO%2B6EeS0iyL_KHffA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHJay-pvs%3D4QoOyfYs%3DZkC_fQTtedO%2B6EeS0iyL_KHffA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5F90E219-EAF2-41AA-848D-99D27CB0952F%40gmail.com
https://groups.google.com/d/msgid/elasticsearch/5F90E219-EAF2-41AA-848D-99D27CB0952F%40gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f143fb0f-625f-4965-8416-346017080b0c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

jprante · December 5, 2014, 7:25pm

Updates/deletes go to primary shard first, it is not possible that they can
fail for one client. Another thing is bulk request action order. ES does
not reorder or sort actions, this is up to the client.

Jörg
Am 04.12.2014 20:38 schrieb mansishah@maprtech.com:

I have a related question about synchronous behavior of Java APIs. I
understand refresh will make sure that the indexed document becomes
searchable but what about subsequent updates / deletes to the same document.

If I index a document and it is going to be done asynchronously does that
mean an immediate update or delete on this document could potentially fail
because the document is not yet actually saved? Note that the update/
delete could be coming from potentially different client but still
serially.

Is there a API like "flush" which guarantees that everything that has been
indexed so far is now available in ES (not for search but for
get/update/delete) at least in the trans log.

On Sunday, April 13, 2014 8:34:19 AM UTC-7, Jörg Prante wrote:

There is massive impact - but try and see for yourself.

Jörg

On Sun, Apr 13, 2014 at 4:54 PM, Edward Chin echi...@gmail.com wrote:

If I do that, and there are concurrent threads doing the same thing, is
there impact to threads that are simply searching? I don't mind the
indexing threads being slow, but i don't want to impact the search threads
too much.

thanks
ed

On Apr 13, 2014, at 10:47 AM, joerg...@gmail.com wrote:

Look at IndexRequestBuilder, there is a method setRefresh(true) so you
can enforce a post refresh per index request.

Jörg

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/elasticsearch/DWN30UYzeN4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/CAKdsXoHJay-pvs%3D4QoOyfYs%3DZkC_fQTtedO%2B6EeS0iyL_
KHffA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHJay-pvs%3D4QoOyfYs%3DZkC_fQTtedO%2B6EeS0iyL_KHffA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/5F90E219-EAF2-41AA-848D-99D27CB0952F%40gmail.com
https://groups.google.com/d/msgid/elasticsearch/5F90E219-EAF2-41AA-848D-99D27CB0952F%40gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f143fb0f-625f-4965-8416-346017080b0c%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/f143fb0f-625f-4965-8416-346017080b0c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFJN%3DocBc%2B9gV6wzi2v94C5kKYAcAmTMQ%2BZe2MyW79Q9Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Java API synchronous vs asynchronous?

thanks Ed

thanks
Ed