Error connecting to ElasticSearch with Zookeeper

Elastic Stack Elasticsearch
1

Hello,

I am new to Elasticsearch and currently doing some tests for considering Elasticsearch as our datastore for the next project.

I have setup zookeeper with Elasticsearch. I have a few VMs running on separate machines in different countries.

On each of these, I have got Elasticsearch running. There is a zookeeper instance running on each of these VMs as well.

I have configured elasticsearch.yml based on this: https://github.com/sonian/elasticsearch-zookeeper

elasticsearch version: 0.19.6
(Version 0.20.0 does not seem to be able to work with zookeeper, hence I am using 0.19.6)

At this point, all the nodes are able to detect a single master. Good so far.

Now, I try to connect to the cluster using TransportClient in Java:

Some code snippets:-

settings = ImmutableSettings.settingsBuilder()
.put("client.transport.sniff", true).build();
(tried setting client.transport.sniff to false, but did not help)
tClient = new TransportClient(settings)
.addTransportAddress(new InetSocketTransportAddress(hostname, 9300));

(I tried giving the ip address instead of the host name, but made no difference)

I am trying a bulk indexing, and keep getting the error:

Jan 21, 2013 5:48:13 PM org.elasticsearch.common.logging.slf4j.Slf4jESLogger internalInfo INFO: failed to get local cluster state for [#transport#-1][inet[/-ip-address-:9300]], disconnecting... org.elasticsearch.transport.ReceiveTimeoutTransportException: [][inet[/-ip-addr-:9300]][cluster/state] request_id [0] timed out after [5002ms] at org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:342) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:680)

org.elasticsearch.client.transport.NoNodeAvailableException: No node available

If I do a health check from command line or do an XPUT/XGET from terminal, it works fine. Its just from the Java client that I cannot get connectivity to work.

A side note: If I do not use zookeeper, I am able to get indexing working. But master detection seems to fail and each VM starts electing itself as the master.

I would like to use the zookeeper to avoid the split brain situation in our project.

Please suggest any ideas on how to get the Java client talk to my cluster.

Please let me know if I am missing something.

Thanks in advance for the help.

Regards!

2

scole wrote:

I have setup zookeeper with Elasticsearch. I have a few VMs running on
separate machines in different countries.

On each of these, I have got Elasticsearch running. There is a zookeeper
instance running on each of these VMs as well.

[...]

A side note: If I do not use zookeeper, I am able to get indexing
working. But master detection seems to fail and each VM starts
electing itself as the master.

I would like to use the zookeeper to avoid the split brain
situation in our project.

Neither is likely to help much with the kind of latency you're
seeing. Have you tried increasing your ping timeouts for ZooKeeper?
I believe by default it expects a short hop between nodes.

-Drew

--

3

Hi Drew,

Thanks for replying. Sorry I could not write back earlier.

I managed to get the zen discovery working with seven nodes running on a VM
each across two countries.

I'm using the Java Transport Client to connect to the cluster and index
data.

The purpose is to measure the time Elasticsearch takes to write bulk data.

I've done similar write performance tests with SOLR and found it's write
performance much higher than Elasticsearch.

Setup: 4 threads, Bulk count: 6000
SOLR: more than 20000 writes per second
Elasticsearch: usually between the 5000 and 7000 mark.

I tried changing the parameters like bulk count, thread count, nodes per VM
etc to see if Elasticsearch write speed improved.

But it did not help.

I have disabled multicast and using unicast in all the conf files (and
added the host names in the discovery.zen.ping.unicast.hosts section.

Sorry for my lack of experience with Elasticsearch, but please can anyone
guide me how I can get higher write speeds with Elasticsearch?

Thanks very much.

On Monday, January 21, 2013 5:56:25 PM UTC, SCole wrote:

Hello,

I am new to Elasticsearch and currently doing some tests for considering
Elasticsearch as our datastore for the next project.

I have setup zookeeper with Elasticsearch. I have a few VMs running on
separate machines in different countries.

On each of these, I have got Elasticsearch running. There is a zookeeper
instance running on each of these VMs as well.

I have configured elasticsearch.yml based on this:
GitHub - sonian/elasticsearch-zookeeper

elasticsearch version: 0.19.6
(Version 0.20.0 does not seem to be able to work with zookeeper, hence I
am
using 0.19.6)

At this point, all the nodes are able to detect a single master. Good so
far.

Now, I try to connect to the cluster using TransportClient in Java:

Some code snippets:-

settings = ImmutableSettings.settingsBuilder()
.put("client.transport.sniff", true).build();
(tried setting client.transport.sniff to false, but did not help)
tClient = new TransportClient(settings)
.addTransportAddress(new
InetSocketTransportAddress(hostname, 9300));

(I tried giving the ip address instead of the host name, but made no
difference)

I am trying a bulk indexing, and keep getting the error:

Jan 21, 2013 5:48:13 PM
org.elasticsearch.common.logging.slf4j.Slf4jESLogger
internalInfo
INFO: failed to get local cluster state for
[#transport#-1][inet[/-ip-address-:9300]], disconnecting...
org.elasticsearch.transport.ReceiveTimeoutTransportException:
[inet[/-ip-addr-:9300]][cluster/state] request_id [0] timed out after
[5002ms]
at
org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:342)

    at

java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

    at

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

    at java.lang.Thread.run(Thread.java:680)

org.elasticsearch.client.transport.NoNodeAvailableException: No node
available*

If I do a health check from command line or do an XPUT/XGET from terminal,
it works fine. Its just from the Java client that I cannot get
connectivity
to work.

A side note: If I do not use zookeeper, I am able to get indexing working.
But master detection seems to fail and each VM starts electing itself as
the
master.

I would like to use the zookeeper to avoid the split brain situation in
our
project.

Please suggest any ideas on how to get the Java client talk to my cluster.

Please let me know if I am missing something.

Thanks in advance for the help.

Regards!

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Error-connecting-to-ElasticSearch-with-Zookeeper-tp4028530.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

--