Java API: How to detect just submitted docs are indexed or not?

Elastic Stack Elasticsearch
1

Hey, I'm just started play with ES and want to try it in a desktop app
using java API.

I noticed that if I submit a set of documents with
client.prepareIndex(indexName, indexType).setSource( /* jsonBuilder stuff
here */ ).execute().actionGet();

then if I try to search right after I submited my docs:
SearchResponse response =
client.prepareSearch(indexName).setQuery(queryString("google")).execute().actionGet();
I get usually no hits, but sometimes I get one hit. Anyway this is not that
I expect: I have 12 documents matching the query.

But if I put this line before the search statement:
Thread.sleep(1000);
I get all my 12 hits.

So the question is how can I know exactly if the index is ready to be
searched?

2

On development mode, try the code below, but I'm not sure if its advisable
in prod since server(fast) should be avail to make your data available in
(near) real time.

client.admin().indices().refresh(new RefreshRequest(indexName)).actionGet();

On Thu, Apr 19, 2012 at 5:53 PM, Diomas dmitriy.skopa@gmail.com wrote:

Hey, I'm just started play with ES and want to try it in a desktop app
using java API.

I noticed that if I submit a set of documents with
client.prepareIndex(indexName, indexType).setSource( /* jsonBuilder stuff
here */ ).execute().actionGet();

then if I try to search right after I submited my docs:
SearchResponse response =
client.prepareSearch(indexName).setQuery(queryString("google")).execute().actionGet();
I get usually no hits, but sometimes I get one hit. Anyway this is not
that I expect: I have 12 documents matching the query.

But if I put this line before the search statement:
Thread.sleep(1000);
I get all my 12 hits.

So the question is how can I know exactly if the index is ready to be
searched?

3

That works well. Thanks! I'm working on a desktop tool (single node, no
shards or replicas) - is it normal for this case?

On Thursday, April 19, 2012 2:12:14 PM UTC+4, Ariel Mendoza wrote:

On development mode, try the code below, but I'm not sure if its advisable
in prod since server(fast) should be avail to make your data available in
(near) real time.

client.admin().indices().refresh(new RefreshRequest(indexName
)).actionGet();

On Thu, Apr 19, 2012 at 5:53 PM, Diomas wrote:

Hey, I'm just started play with ES and want to try it in a desktop app
using java API.

I noticed that if I submit a set of documents with
client.prepareIndex(indexName, indexType).setSource( /* jsonBuilder stuff
here */ ).execute().actionGet();

then if I try to search right after I submited my docs:
SearchResponse response =
client.prepareSearch(indexName).setQuery(queryString("google")).execute().actionGet();
I get usually no hits, but sometimes I get one hit. Anyway this is not
that I expect: I have 12 documents matching the query.

But if I put this line before the search statement:
Thread.sleep(1000);
I get all my 12 hits.

So the question is how can I know exactly if the index is ready to be
searched?

On Thursday, April 19, 2012 2:12:14 PM UTC+4, Ariel Mendoza wrote:

On development mode, try the code below, but I'm not sure if its advisable
in prod since server(fast) should be avail to make your data available in
(near) real time.

client.admin().indices().refresh(new RefreshRequest(indexName
)).actionGet();

On Thu, Apr 19, 2012 at 5:53 PM, Diomas wrote:

Hey, I'm just started play with ES and want to try it in a desktop app
using java API.

I noticed that if I submit a set of documents with
client.prepareIndex(indexName, indexType).setSource( /* jsonBuilder stuff
here */ ).execute().actionGet();

then if I try to search right after I submited my docs:
SearchResponse response =
client.prepareSearch(indexName).setQuery(queryString("google")).execute().actionGet();
I get usually no hits, but sometimes I get one hit. Anyway this is not
that I expect: I have 12 documents matching the query.

But if I put this line before the search statement:
Thread.sleep(1000);
I get all my 12 hits.

So the question is how can I know exactly if the index is ready to be
searched?

4

I guess your using Node client. You can shift to transport and set the
number of shards and replicas that your desktop can handle.

  1. In {ES_HOME}/config/elasticsearch.yml edit the property below and leave
    everything else unchanged. By default it has 5 shards and 1 replica each.
    cluster.name: MyIndex

  2. Java API Transport Client:
    Settings settings = ImmutableSettings.settingsBuilder()
    .put("cluster.name", "MyIndex")
    .put("client.transport.sniff", true)
    .build();

         TransportClient client = new TransportClient(settings);
     client.addTransportAddress(new

InetSocketTransportAddress("localhost", 9300));
client.addTransportAddress(new
InetSocketTransportAddress("localhost", 9301));

  1. You can have multiple es instance in one machine. Open a two(2) command
    line(windows) and execute {ES_HOME}/bin/elasticsearch.bat in each. This
    should the run transports addresses; localhost:9300 and localhost:9301.
    Same is true if you have multiple machine.

  2. Download elasticsearch-head to ease up your monitoring of indexed data
    and/or debugging.

Enjoy!

On Thu, Apr 19, 2012 at 6:54 PM, Diomas dmitriy.skopa@gmail.com wrote:

That works well. Thanks! I'm working on a desktop tool (single node, no
shards or replicas) - is it normal for this case?

On Thursday, April 19, 2012 2:12:14 PM UTC+4, Ariel Mendoza wrote:

On development mode, try the code below, but I'm not sure if its
advisable in prod since server(fast) should be avail to make your data
available in (near) real time.

client.admin().indices().refresh(new RefreshRequest(indexName
)).actionGet();

On Thu, Apr 19, 2012 at 5:53 PM, Diomas wrote:

Hey, I'm just started play with ES and want to try it in a desktop app
using java API.

I noticed that if I submit a set of documents with
client.prepareIndex(indexName, indexType).setSource( /* jsonBuilder
stuff here */ ).execute().actionGet();

then if I try to search right after I submited my docs:
SearchResponse response =
client.prepareSearch(indexName).setQuery(queryString("google")).execute().actionGet();
I get usually no hits, but sometimes I get one hit. Anyway this is not
that I expect: I have 12 documents matching the query.

But if I put this line before the search statement:
Thread.sleep(1000);
I get all my 12 hits.

So the question is how can I know exactly if the index is ready to be
searched?

On Thursday, April 19, 2012 2:12:14 PM UTC+4, Ariel Mendoza wrote:

On development mode, try the code below, but I'm not sure if its
advisable in prod since server(fast) should be avail to make your data
available in (near) real time.

client.admin().indices().refresh(new RefreshRequest(indexName
)).actionGet();

On Thu, Apr 19, 2012 at 5:53 PM, Diomas wrote:

Hey, I'm just started play with ES and want to try it in a desktop app
using java API.

I noticed that if I submit a set of documents with
client.prepareIndex(indexName, indexType).setSource( /* jsonBuilder
stuff here */ ).execute().actionGet();

then if I try to search right after I submited my docs:
SearchResponse response =
client.prepareSearch(indexName).setQuery(queryString("google")).execute().actionGet();
I get usually no hits, but sometimes I get one hit. Anyway this is not
that I expect: I have 12 documents matching the query.

But if I put this line before the search statement:
Thread.sleep(1000);
I get all my 12 hits.

So the question is how can I know exactly if the index is ready to be
searched?

5

When you index a document, its indexed, but, it won't be immediately
available to search (thats the near real time aspect). by default, there is
a refresh_interval of 1 second where a the index will be refreshed every
second. You can issue a refresh API call yourself, but its not recommended
to do it after each index request.

On Thu, Apr 19, 2012 at 1:54 PM, Diomas dmitriy.skopa@gmail.com wrote:

That works well. Thanks! I'm working on a desktop tool (single node, no
shards or replicas) - is it normal for this case?

On Thursday, April 19, 2012 2:12:14 PM UTC+4, Ariel Mendoza wrote:

On development mode, try the code below, but I'm not sure if its
advisable in prod since server(fast) should be avail to make your data
available in (near) real time.

client.admin().indices().refresh(new RefreshRequest(indexName
)).actionGet();

On Thu, Apr 19, 2012 at 5:53 PM, Diomas wrote:

Hey, I'm just started play with ES and want to try it in a desktop app
using java API.

I noticed that if I submit a set of documents with
client.prepareIndex(indexName, indexType).setSource( /* jsonBuilder
stuff here */ ).execute().actionGet();

then if I try to search right after I submited my docs:
SearchResponse response =
client.prepareSearch(indexName).setQuery(queryString("google")).execute().actionGet();
I get usually no hits, but sometimes I get one hit. Anyway this is not
that I expect: I have 12 documents matching the query.

But if I put this line before the search statement:
Thread.sleep(1000);
I get all my 12 hits.

So the question is how can I know exactly if the index is ready to be
searched?

On Thursday, April 19, 2012 2:12:14 PM UTC+4, Ariel Mendoza wrote:

On development mode, try the code below, but I'm not sure if its
advisable in prod since server(fast) should be avail to make your data
available in (near) real time.

client.admin().indices().refresh(new RefreshRequest(indexName
)).actionGet();

On Thu, Apr 19, 2012 at 5:53 PM, Diomas wrote:

Hey, I'm just started play with ES and want to try it in a desktop app
using java API.

I noticed that if I submit a set of documents with
client.prepareIndex(indexName, indexType).setSource( /* jsonBuilder
stuff here */ ).execute().actionGet();

then if I try to search right after I submited my docs:
SearchResponse response =
client.prepareSearch(indexName).setQuery(queryString("google")).execute().actionGet();
I get usually no hits, but sometimes I get one hit. Anyway this is not
that I expect: I have 12 documents matching the query.

But if I put this line before the search statement:
Thread.sleep(1000);
I get all my 12 hits.

So the question is how can I know exactly if the index is ready to be
searched?