# Need help with Java API

**URL:** <https://discuss.elastic.co/t/need-help-with-java-api/18768>\
**Category:** Elasticsearch\
**Created:** [July 21, 2014, 4:53pm UTC](https://discuss.elastic.co/t/need-help-with-java-api/18768 "2014-07-21T16:53:17Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Alain\_Desilets](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alain_desilets/32/29314_2.png) [@Alain\_Desilets](https://discuss.elastic.co/u/Alain_Desilets)\
**Post date:** [July 21, 2014, 4:53pm UTC](https://discuss.elastic.co/t/need-help-with-java-api/18768/1 "2014-07-21T16:53:17Z")

</div>

I am trying to get started with the Java API, using the excellent tutorial  
found here:

[http://www.slideshare.net/dadoonet/hands-on-lab-elasticsearch](http://www.slideshare.net/dadoonet/hands-on-lab-elasticsearch)

But I am still having a lot of trouble.

Below is a sample of code that I have written:

package ca.nrc.ElasticSearch;

import org.codehaus.jackson.map.ObjectMapper;  
import org.elasticsearch.action.get.GetResponse;  
import org.elasticsearch.action.index.IndexResponse;  
import org.elasticsearch.client.Client;  
import org.elasticsearch.node.NodeBuilder;

public class ElasticSearchRunner {

static ObjectMapper mapper;  
static Client client;  
static String indexName = "meal5";  
static String typeName = "beer";  
static long startTimeMSecs;

public static void main(String[] args) throws Exception {  
startTimeMSecs = System.currentTimeMillis();  
mapper = new ObjectMapper(); // create once, reuse  
echo("Creating the ElasticSearch client...");  
client = NodeBuilder.nodeBuilder().node().client(); // Does this create a  
brand new cluster?  
// client =  
NodeBuilder.nodeBuilder().clusterName("handson").client(true).node().client();  
// Joins existing cluster called "handson"  
echo("DONE creating the ElasticSearch client... Elapsed time =  
"+elapsedSecs()+" secs.");  
echo("Creating a beer object...");  
Beer beer = new Beer("Heineken", Colour.PALE, 0.33, 3);  
String jsonString = mapper.writeValueAsString(beer);  
echo("DONE Creating a beer object...");

echo("Indexing the beer object...");  
IndexResponse ir = null;  
ir = client.prepareIndex(indexName, typeName).setSource(jsonString)  
.execute().actionGet();  
echo("DONE Indexing the beer object...");

echo("Retrieving the beer object...");  
GetResponse gr = null;  
gr = client.prepareGet(indexName, typeName, ir.getId()).execute()  
.actionGet();  
echo("DONE Retrieving the beer object...");  
}

public static float elapsedSecs() {  
float elapsed = (System.currentTimeMillis() - startTimeMSecs)/1000;  
return elapsed;  
}  
public static void echo(String mess) {  
mess = mess + " (Elapsed so far: "+elapsedSecs()+" seconds)";  
System.out.println(mess);  
}  
}

It works, "sort of"...

If I use the first method for creating the client:

client = NodeBuilder.nodeBuilder().node().client();

Then it works fin the first time I run it. However:

\*\*\* ISSUE 1: If I try to inspect the meal index with Marvel, I don't find  
it.

Also,

\*\*\* ISSUE 2: If I run the application a second time, I get the following  
output:

Creating the ElasticSearch client... (Elapsed so far: 0.0 seconds)  
log4j:WARN No appenders could be found for logger (org.elasticsearch.node).  
log4j:WARN Please initialize the log4j system properly.  
log4j:WARN See [http://logging.apache.org/log4j/1.2/faq.html#noconfig](http://logging.apache.org/log4j/1.2/faq.html#noconfig) for  
more info.  
DONE creating the ElasticSearch client... Elapsed time = 9.0 secs. (Elapsed  
so far: 9.0 seconds)  
Creating a beer object... (Elapsed so far: 9.0 seconds)  
DONE Creating a beer object... (Elapsed so far: 9.0 seconds)  
Indexing the beer object... (Elapsed so far: 9.0 seconds)  
Exception in thread "main"  
org.elasticsearch.action.UnavailableShardsException: [meal5][0] [2]  
shardIt, [0] active : Timeout waiting for [1m], request: index  
{[meal5][beer][B3F5ZEmSTruqdnlxhYviFg],  
source[{"brand":"Heineken","colour":"PALE","size":0.33,"price":3.0}]}  
at  
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.raiseTimeoutFailure(TransportShardReplicationOperationAction.java:526)  
at  
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$3.onTimeout(TransportShardReplicationOperationAction.java:516)  
at  
org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:239)  
at  
org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:494)  
at  
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)  
at  
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)  
at java.lang.Thread.run(Thread.java:722)

For me to be able to run the application again, I have to change the  
indexName variable to a different value (ex: "meal6").

Also:

\*\*\* ISSUE 3: If I try using the second method for creating a Client, then  
it doesn't work. More specifically, if I use this method:

client =  
NodeBuilder.nodeBuilder().clusterName("handson").client(true).node().client();

This yields:

Creating the ElasticSearch client... (Elapsed so far: 0.0 seconds)  
log4j:WARN No appenders could be found for logger (org.elasticsearch.node).  
log4j:WARN Please initialize the log4j system properly.  
log4j:WARN See [http://logging.apache.org/log4j/1.2/faq.html#noconfig](http://logging.apache.org/log4j/1.2/faq.html#noconfig) for  
more info.  
DONE creating the ElasticSearch client... Elapsed time = 34.0 secs.  
(Elapsed so far: 34.0 seconds)  
Creating a beer object... (Elapsed so far: 34.0 seconds)  
DONE Creating a beer object... (Elapsed so far: 34.0 seconds)  
Indexing the beer object... (Elapsed so far: 34.0 seconds)  
Exception in thread "main"  
org.elasticsearch.discovery.MasterNotDiscoveredException: waited for [1m]  
at  
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$4.onTimeout(TransportMasterNodeOperationAction.java:170)  
at  
org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:239)  
at  
org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:494)  
at  
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)  
at  
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)  
at java.lang.Thread.run(Thread.java:722)

Yet, the "handson" cluster does exist and I can see it in Marvel.

Finally, I am puzzled by the following:

\*\*\* ISSUE 4: It seems that it takes ES 8-9 secs to create a Client using  
the first method, and \>30 secs with the second method. That seems really  
high. Is that normal?

Any help you can provide with either of those issues will be greatly  
appreciated. Thx.

Alain

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/ba7e38a6-fb11-4a4d-95b6-4b0f2c904061%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/ba7e38a6-fb11-4a4d-95b6-4b0f2c904061%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [July 21, 2014, 5:13pm UTC](https://discuss.elastic.co/t/need-help-with-java-api/18768/2 "2014-07-21T17:13:47Z")

</div>

To issue 1: you create a single node cluster without index, and a client of  
it.

To issue 2: you see the UnavailableShardsException caused by a timeout  
while indexing to a replica shard. This means, you may have set up a single  
node cluster, but with replica level 1 (default) which needs 2 nodes for  
indexing. Maybe there was once another node joining the cluster and ES  
wants it back abut it never came (after 60 secs). Then ES returns the  
timeout error. Maybe replica level 0 helps. You should also check the  
cluster health. A color of green shows everything works, yellow means there  
are too few nodes to satisfy the replica condition, and read means the  
cluster is not in a consistent/usable state.

To issue 3: not sure what clusterName() means. I would use settings and add  
a field "cluster.name". Maybe it is equivalent. You must ensure you use the  
same "cluster.name" setting throughout all nodes and clients. You also can  
not reuse data from clusters that had other names (look into the "data"  
folder)

To issue 4: ES takes ~5 secs for discovery, the zen modules pings and waits  
for responding master nodes by default. If you just test locally on your  
developer machine, you should disable zen. Most easily by disabling  
networking at all, by using NodeBuilder.nodeBuilder().local(true)...

Jörg

On Mon, Jul 21, 2014 at 6:53 PM, Alain Désilets [alaindesilets0@gmail.com](mailto:alaindesilets0@gmail.com)  
wrote:

> I am trying to get started with the Java API, using the excellent tutorial  
> found here:
> 
> [Hands on lab Elasticsearch | PPT](http://www.slideshare.net/dadoonet/hands-on-lab-elasticsearch)
> 
> But I am still having a lot of trouble.
> 
> Below is a sample of code that I have written:
> 
> package ca.nrc.Elasticsearch;
> 
> import org.codehaus.jackson.map.ObjectMapper;  
> import org.elasticsearch.action.get.GetResponse;  
> import org.elasticsearch.action.index.IndexResponse;  
> import org.elasticsearch.client.Client;  
> import org.elasticsearch.node.NodeBuilder;
> 
> public class ElasticSearchRunner {
> 
> static ObjectMapper mapper;  
> static Client client;  
> static String indexName = "meal5";  
> static String typeName = "beer";  
> static long startTimeMSecs;
> 
> public static void main(String args) throws Exception {  
> startTimeMSecs = System.currentTimeMillis();  
> mapper = new ObjectMapper(); // create once, reuse  
> echo("Creating the Elasticsearch client...");  
> client = NodeBuilder.nodeBuilder().node().client(); // Does this create a  
> brand new cluster?  
> // client =  
> NodeBuilder.nodeBuilder().clusterName("handson").client(true).node().client();  
> // Joins existing cluster called "handson"  
> echo("DONE creating the Elasticsearch client... Elapsed time =  
> "+elapsedSecs()+" secs.");  
> echo("Creating a beer object...");  
> Beer beer = new Beer("Heineken", Colour.PALE, 0.33, 3);  
> String jsonString = mapper.writeValueAsString(beer);  
> echo("DONE Creating a beer object...");
> 
> echo("Indexing the beer object...");  
> IndexResponse ir = null;  
> ir = client.prepareIndex(indexName, typeName).setSource(jsonString)  
> .execute().actionGet();  
> echo("DONE Indexing the beer object...");
> 
> echo("Retrieving the beer object...");  
> GetResponse gr = null;  
> gr = client.prepareGet(indexName, typeName, ir.getId()).execute()  
> .actionGet();  
> echo("DONE Retrieving the beer object...");  
> }
> 
> public static float elapsedSecs() {  
> float elapsed = (System.currentTimeMillis() - startTimeMSecs)/1000;  
> return elapsed;  
> }  
> public static void echo(String mess) {  
> mess = mess + " (Elapsed so far: "+elapsedSecs()+" seconds)";  
> System.out.println(mess);  
> }  
> }
> 
> It works, "sort of"...
> 
> If I use the first method for creating the client:
> 
> client = NodeBuilder.nodeBuilder().node().client();
> 
> Then it works fin the first time I run it. However:
> 
> \*\*\* ISSUE 1: If I try to inspect the meal index with Marvel, I don't find  
> it.
> 
> Also,
> 
> \*\*\* ISSUE 2: If I run the application a second time, I get the following  
> output:
> 
> Creating the Elasticsearch client... (Elapsed so far: 0.0 seconds)  
> log4j:WARN No appenders could be found for logger (org.elasticsearch.node).  
> log4j:WARN Please initialize the log4j system properly.  
> log4j:WARN See [Apache log4j 1.2 - Frequently Asked Technical Questions](http://logging.apache.org/log4j/1.2/faq.html#noconfig) for  
> more info.  
> DONE creating the Elasticsearch client... Elapsed time = 9.0 secs.  
> (Elapsed so far: 9.0 seconds)  
> Creating a beer object... (Elapsed so far: 9.0 seconds)  
> DONE Creating a beer object... (Elapsed so far: 9.0 seconds)  
> Indexing the beer object... (Elapsed so far: 9.0 seconds)  
> Exception in thread "main"  
> org.elasticsearch.action.UnavailableShardsException: [meal5][0] [2]  
> shardIt, [0] active : Timeout waiting for [1m], request: index  
> {[meal5][beer][B3F5ZEmSTruqdnlxhYviFg],  
> source[{"brand":"Heineken","colour":"PALE","size":0.33,"price":3.0}]}  
> at  
> org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.raiseTimeoutFailure(TransportShardReplicationOperationAction.java:526)  
> at  
> org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$3.onTimeout(TransportShardReplicationOperationAction.java:516)  
> at  
> org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:239)  
> at  
> org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:494)  
> at  
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)  
> at  
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)  
> at java.lang.Thread.run(Thread.java:722)
> 
> For me to be able to run the application again, I have to change the  
> indexName variable to a different value (ex: "meal6").
> 
> Also:
> 
> \*\*\* ISSUE 3: If I try using the second method for creating a Client, then  
> it doesn't work. More specifically, if I use this method:
> 
> client =  
> NodeBuilder.nodeBuilder().clusterName("handson").client(true).node().client();
> 
> This yields:
> 
> Creating the Elasticsearch client... (Elapsed so far: 0.0 seconds)  
> log4j:WARN No appenders could be found for logger (org.elasticsearch.node).  
> log4j:WARN Please initialize the log4j system properly.  
> log4j:WARN See [Apache log4j 1.2 - Frequently Asked Technical Questions](http://logging.apache.org/log4j/1.2/faq.html#noconfig) for  
> more info.  
> DONE creating the Elasticsearch client... Elapsed time = 34.0 secs.  
> (Elapsed so far: 34.0 seconds)  
> Creating a beer object... (Elapsed so far: 34.0 seconds)  
> DONE Creating a beer object... (Elapsed so far: 34.0 seconds)  
> Indexing the beer object... (Elapsed so far: 34.0 seconds)  
> Exception in thread "main"  
> org.elasticsearch.discovery.MasterNotDiscoveredException: waited for [1m]  
> at  
> org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$4.onTimeout(TransportMasterNodeOperationAction.java:170)  
> at  
> org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:239)  
> at  
> org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:494)  
> at  
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)  
> at  
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)  
> at java.lang.Thread.run(Thread.java:722)
> 
> Yet, the "handson" cluster does exist and I can see it in Marvel.
> 
> Finally, I am puzzled by the following:
> 
> \*\*\* ISSUE 4: It seems that it takes ES 8-9 secs to create a Client using  
> the first method, and \>30 secs with the second method. That seems really  
> high. Is that normal?
> 
> Any help you can provide with either of those issues will be greatly  
> appreciated. Thx.
> 
> Alain
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/ba7e38a6-fb11-4a4d-95b6-4b0f2c904061%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/ba7e38a6-fb11-4a4d-95b6-4b0f2c904061%40googlegroups.com)  
> [https://groups.google.com/d/msgid/elasticsearch/ba7e38a6-fb11-4a4d-95b6-4b0f2c904061%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/ba7e38a6-fb11-4a4d-95b6-4b0f2c904061%40googlegroups.com?utm_medium=email&utm_source=footer)  
> .  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGYALxCux0iZ749UhWM7gBRHRubCP2zcC7RT2o0eCWDdg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGYALxCux0iZ749UhWM7gBRHRubCP2zcC7RT2o0eCWDdg%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Alain\_Desilets](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alain_desilets/32/29314_2.png) [@Alain\_Desilets](https://discuss.elastic.co/u/Alain_Desilets)\
**Post date:** [July 22, 2014, 6:00pm UTC](https://discuss.elastic.co/t/need-help-with-java-api/18768/3 "2014-07-22T18:00:33Z")

</div>

Thx Jörg,

Your comments helped me get quite a bit further. Here is an updated version  
of my code:

[Demos how to use the ElasticSearch Java API · GitHub](https://gist.github.com/alaindesilets/aec9492890c37075fa4e)

On Monday, 21 July 2014 13:13:58 UTC-4, Jörg Prante wrote:

> To issue 1: you create a single node cluster without index, and a client  
> of it.

Duh! Don't know how I could have missed that. So now, I added a  
method createIndex() which creates the index if it doesn't exist, and sets  
number\_of\_replicas to 2. It doesn't define mappings since the ES doc says  
that default mappings will be automatically generated if none are specified.

Note however that even with that change, I still couldn't see the new index  
in Marvel/Sense, when I instantiated the client with  
method makeClientFromEmbeddedNode(). But if I instantiate the client  
through a TransportClient, i.e. by invoking  
method makeClientFromTransportClient(), then Marvel/Sense sees the new  
index.

Using a TransportClient instead of a client obtained from an embedded node  
also greatly accelerated the client creation. Instead of taking \> 8 secs to  
create the client, it takes \< 1 sec. Not sure why.

> To issue 2: you see the UnavailableShardsException caused by a timeout  
> while indexing to a replica shard. This means, you may have set up a single  
> node cluster, but with replica level 1 (default) which needs 2 nodes for  
> indexing. Maybe there was once another node joining the cluster and ES  
> wants it back abut it never came (after 60 secs). Then ES returns the  
> timeout error. Maybe replica level 0 helps. You should also check the  
> cluster health. A color of green shows everything works, yellow means there  
> are too few nodes to satisfy the replica condition, and read means the  
> cluster is not in a consistent/usable state.

I tried setting number\_replica=0 and number\_replica=1 in createIndex(), but  
I still got the org.elasticsearch.action.UnavailableShardsException error  
if I instantiate the client with makeClientFromEmbeddedNode(). But I don't  
get the error if I instantiate the client with  
makeClientFromTransportClient(), independantly of the number of replicas I  
specify.

> To issue 3: not sure what clusterName() means. I would use settings and  
> add a field "cluster.name". Maybe it is equivalent. You must ensure you  
> use the same "cluster.name" setting throughout all nodes and clients. You  
> also can not reuse data from clusters that had other names (look into the  
> "data" folder)

I have now moved that code to a method called  
makeClientFromNamedClusterNode(). I haven't been able to make that work at  
all.

To issue 4: ES takes ~5 secs for discovery, the zen modules pings and waits

> for responding master nodes by default. If you just test locally on your  
> developer machine, you should disable zen. Most easily by disabling  
> networking at all, by using NodeBuilder.nodeBuilder().local(true)...

Not sure I understand what that means, but in any case, using a  
TransportClient seems to address that issue.

So, all in all, I feel I am in pretty good shape now. Thanks for the help.

There is one new Issue I an now encountering, when I try to do a bulk  
indexing, namely:

Issue 5: If I uncomment the call to index100NewBeersInOneBatch(), I get the  
following:

== Indexing 100 new beer objects as in one batch... (Elapsed so far: 1.0  
seconds)  
Adding beer no 0 to the bulk request. (Elapsed so far: 1.0 seconds)  
Exception in thread "main" java.lang.NullPointerException  
at org.elasticsearch.action.bulk.BulkRequest.internalAdd(BulkRequest  
.java:129)  
at org.elasticsearch.action.bulk.BulkRequest.add(BulkRequest.java:  
118)  
at org.elasticsearch.action.bulk.BulkRequestBuilder.add(  
BulkRequestBuilder.java:52)  
at ca.nrc.Elasticsearch.ElasticSearchDemo.index100NewBeersInOneBatch  
(ElasticSearchDemo.java:158)  
at ca.nrc.Elasticsearch.ElasticSearchDemo.main(ElasticSearchDemo.  
java:72)

Any thoughts on what might be going on there?

Thx.

Alain

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/c3f6c329-4dc4-4714-8e6b-bae1b7fa1f51%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c3f6c329-4dc4-4714-8e6b-bae1b7fa1f51%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [July 22, 2014, 6:21pm UTC](https://discuss.elastic.co/t/need-help-with-java-api/18768/4 "2014-07-22T18:21:22Z")

</div>

To Issue 5: unfortunately, there are some hidden NPEs in the bulk request  
API.

Check this

[https://github.com/jprante/elasticsearch-support/blob/master/src/test/java/org/xbib/elasticsearch/support/various/NPETest.java](https://github.com/jprante/elasticsearch-support/blob/master/src/test/java/org/xbib/elasticsearch/support/various/NPETest.java)

if you find a pattern that matches your case.

Jörg

On Tue, Jul 22, 2014 at 8:00 PM, Alain Désilets [alaindesilets0@gmail.com](mailto:alaindesilets0@gmail.com)  
wrote:

> Thx Jörg,
> 
> Your comments helped me get quite a bit further. Here is an updated  
> version of my code:
> 
> [Demos how to use the ElasticSearch Java API · GitHub](https://gist.github.com/alaindesilets/aec9492890c37075fa4e)
> 
> On Monday, 21 July 2014 13:13:58 UTC-4, Jörg Prante wrote:
> 
> > To issue 1: you create a single node cluster without index, and a client  
> > of it.
> 
> Duh! Don't know how I could have missed that. So now, I added a  
> method createIndex() which creates the index if it doesn't exist, and sets  
> number\_of\_replicas to 2. It doesn't define mappings since the ES doc says  
> that default mappings will be automatically generated if none are specified.
> 
> Note however that even with that change, I still couldn't see the new  
> index in Marvel/Sense, when I instantiated the client with  
> method makeClientFromEmbeddedNode(). But if I instantiate the client  
> through a TransportClient, i.e. by invoking  
> method makeClientFromTransportClient(), then Marvel/Sense sees the new  
> index.
> 
> Using a TransportClient instead of a client obtained from an embedded node  
> also greatly accelerated the client creation. Instead of taking \> 8 secs to  
> create the client, it takes \< 1 sec. Not sure why.
> 
> > To issue 2: you see the UnavailableShardsException caused by a timeout  
> > while indexing to a replica shard. This means, you may have set up a single  
> > node cluster, but with replica level 1 (default) which needs 2 nodes for  
> > indexing. Maybe there was once another node joining the cluster and ES  
> > wants it back abut it never came (after 60 secs). Then ES returns the  
> > timeout error. Maybe replica level 0 helps. You should also check the  
> > cluster health. A color of green shows everything works, yellow means there  
> > are too few nodes to satisfy the replica condition, and read means the  
> > cluster is not in a consistent/usable state.
> 
> I tried setting number\_replica=0 and number\_replica=1 in createIndex(),  
> but I still got the org.elasticsearch.action.UnavailableShardsException  
> error if I instantiate the client with makeClientFromEmbeddedNode(). But I  
> don't get the error if I instantiate the client with  
> makeClientFromTransportClient(), independantly of the number of replicas I  
> specify.
> 
> > To issue 3: not sure what clusterName() means. I would use settings and  
> > add a field "cluster.name". Maybe it is equivalent. You must ensure you  
> > use the same "cluster.name" setting throughout all nodes and clients.  
> > You also can not reuse data from clusters that had other names (look into  
> > the "data" folder)
> 
> I have now moved that code to a method called  
> makeClientFromNamedClusterNode(). I haven't been able to make that work at  
> all.
> 
> To issue 4: ES takes ~5 secs for discovery, the zen modules pings and
> 
> > waits for responding master nodes by default. If you just test locally on  
> > your developer machine, you should disable zen. Most easily by disabling  
> > networking at all, by using NodeBuilder.nodeBuilder().local(true)...
> 
> Not sure I understand what that means, but in any case, using a  
> TransportClient seems to address that issue.
> 
> So, all in all, I feel I am in pretty good shape now. Thanks for the help.
> 
> There is one new Issue I an now encountering, when I try to do a bulk  
> indexing, namely:
> 
> Issue 5: If I uncomment the call to index100NewBeersInOneBatch(), I get  
> the following:
> 
> == Indexing 100 new beer objects as in one batch... (Elapsed so far: 1.0  
> seconds)  
> Adding beer no 0 to the bulk request. (Elapsed so far: 1.0 seconds)  
> Exception in thread "main" java.lang.NullPointerException  
> at org.elasticsearch.action.bulk.BulkRequest.internalAdd(  
> BulkRequest.java:129)  
> at org.elasticsearch.action.bulk.BulkRequest.add(BulkRequest.java:  
> 118)  
> at org.elasticsearch.action.bulk.BulkRequestBuilder.add(  
> BulkRequestBuilder.java:52)  
> at ca.nrc.Elasticsearch.ElasticSearchDemo.  
> index100NewBeersInOneBatch(ElasticSearchDemo.java:158)  
> at ca.nrc.Elasticsearch.ElasticSearchDemo.main(ElasticSearchDemo.  
> java:72)
> 
> Any thoughts on what might be going on there?
> 
> Thx.
> 
> Alain
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/c3f6c329-4dc4-4714-8e6b-bae1b7fa1f51%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c3f6c329-4dc4-4714-8e6b-bae1b7fa1f51%40googlegroups.com)  
> [https://groups.google.com/d/msgid/elasticsearch/c3f6c329-4dc4-4714-8e6b-bae1b7fa1f51%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/c3f6c329-4dc4-4714-8e6b-bae1b7fa1f51%40googlegroups.com?utm_medium=email&utm_source=footer)  
> .
> 
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEpsONV0b40mhtepRs7ksnzQqX\_%2BjPovVPZupOGKiK7JA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEpsONV0b40mhtepRs7ksnzQqX_%2BjPovVPZupOGKiK7JA%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Alain\_Desilets](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alain_desilets/32/29314_2.png) [@Alain\_Desilets](https://discuss.elastic.co/u/Alain_Desilets)\
**Post date:** [July 22, 2014, 6:26pm UTC](https://discuss.elastic.co/t/need-help-with-java-api/18768/5 "2014-07-22T18:26:40Z")

</div>

On Tuesday, 22 July 2014 14:21:29 UTC-4, Jörg Prante wrote:

> To Issue 5: unfortunately, there are some hidden NPEs in the bulk request  
> API.

Ah, OK.

I guess I'll stay away from the bulk request API then.

Alain

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f6a4d5df-52fd-4ad7-899f-964d7b7d451a%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f6a4d5df-52fd-4ad7-899f-964d7b7d451a%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [July 22, 2014, 6:45pm UTC](https://discuss.elastic.co/t/need-help-with-java-api/18768/6 "2014-07-22T18:45:21Z")

</div>

Noo.... bulk request API is fantistic and a must!

Just take a bit care to not pass nulls or empty requests to it.

Jörg

On Tue, Jul 22, 2014 at 8:26 PM, Alain Désilets [alaindesilets0@gmail.com](mailto:alaindesilets0@gmail.com)  
wrote:

> On Tuesday, 22 July 2014 14:21:29 UTC-4, Jörg Prante wrote:
> 
> > To Issue 5: unfortunately, there are some hidden NPEs in the bulk request  
> > API.
> 
> Ah, OK.
> 
> I guess I'll stay away from the bulk request API then.
> 
> Alain
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/f6a4d5df-52fd-4ad7-899f-964d7b7d451a%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f6a4d5df-52fd-4ad7-899f-964d7b7d451a%40googlegroups.com)  
> [https://groups.google.com/d/msgid/elasticsearch/f6a4d5df-52fd-4ad7-899f-964d7b7d451a%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f6a4d5df-52fd-4ad7-899f-964d7b7d451a%40googlegroups.com?utm_medium=email&utm_source=footer)  
> .
> 
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGwsTwOBEaR2ih3fdd%2B6TmVUc7LoVMYrAFpEjOtK-dWkg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGwsTwOBEaR2ih3fdd%2B6TmVUc7LoVMYrAFpEjOtK-dWkg%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Alain\_Desilets](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alain_desilets/32/29314_2.png) [@Alain\_Desilets](https://discuss.elastic.co/u/Alain_Desilets)\
**Post date:** [July 22, 2014, 8:39pm UTC](https://discuss.elastic.co/t/need-help-with-java-api/18768/7 "2014-07-22T20:39:40Z")

</div>

On Tuesday, 22 July 2014 14:45:28 UTC-4, Jörg Prante wrote:

> Noo.... bulk request API is fantistic and a must!
> 
> Just take a bit care to not pass nulls or empty requests to it.

Thx for the encouragement. I perservered a bit more and then realized that  
I had misunderstood how to create an IndexRequest. I fixed the problem and  
it's now working.

I have updated the gist demo code accordingly.

Alain

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f40afe21-42a9-40af-a6a0-0779eb6a9d41%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f40afe21-42a9-40af-a6a0-0779eb6a9d41%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:14am UTC](https://discuss.elastic.co/t/need-help-with-java-api/18768/8 "2017-07-06T01:14:08Z")

</div>