Unable to create a Client node

Elastic Stack Elasticsearch
1

I am trying to use Java API to create an ES Client. My Grails worked fine
utilizing Elasticsearch in the local (same JVM) mode. For clustering
purposes I am trying to change the application to use the "node" mode. The
application uses Grail Elasticsearch plugin. I first start ES server (I am
using 0.19.3 on both ends). After the server starts I start the Grails
application which through the plugin runs the following code:

def nb = NodeBuilder.nodeBuilder()
nb.client(true)
node = nb.node()
node.start()
def client = node.client()
...

The Grails application is unable to start because the new node does not
connect to the server. I can see the node trying to connect.

Here is the server log:

[2012-09-26 13:10:34,023][INFO ][node ] [Sinclar,

Nekra] {0.19.3}[9146]: initializing ...
[2012-09-26 13:10:34,027][INFO ][plugins ] [Sinclar,
Nekra] loaded , sites
[2012-09-26 13:10:35,519][INFO ][node ] [Sinclar,
Nekra] {0.19.3}[9146]: initialized
[2012-09-26 13:10:35,519][INFO ][node ] [Sinclar,
Nekra] {0.19.3}[9146]: starting ...
[2012-09-26 13:10:35,570][INFO ][transport ] [Sinclar,
Nekra] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address
{inet[/10.10.16.4:9300]}
[2012-09-26 13:10:38,608][INFO ][cluster.service ] [Sinclar,
Nekra] new_master [Sinclar,
Nekra][un4UD12IRg2S5aGPoOc-rQ][inet[/10.10.16.4:9300]], reason:
zen-disco-join (elected_as_master)
[2012-09-26 13:10:38,643][INFO ][discovery ] [Sinclar,
Nekra] elasticsearch/un4UD12IRg2S5aGPoOc-rQ
[2012-09-26 13:10:38,652][INFO ][http ] [Sinclar,
Nekra] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address
{inet[/10.10.16.4:9200]}
[2012-09-26 13:10:38,652][INFO ][node ] [Sinclar,
Nekra] {0.19.3}[9146]: started
[2012-09-26 13:10:38,653][INFO ][gateway ] [Sinclar,
Nekra] recovered [0] indices into cluster_state
[2012-09-26 13:11:06,737][INFO ][cluster.service ] [Sinclar,
Nekra] added
{[Nighthawk][dZwzxHghQSir5vESBs-InQ][inet[/10.10.16.4:9301]]{client=true,
data=false},}, reason: zen-disco-receive(join from
node[[Nighthawk][dZwzxHghQSir5vESBs-InQ][inet[/10.10.16.4:9301]]{client=true,
data=false}])
[2012-09-26 13:11:09,748][WARN ][discovery.zen ] [Sinclar,
Nekra] received a join request for an existing node
[[Nighthawk][dZwzxHghQSir5vESBs-InQ][inet[/10.10.16.4:9301]]{client=true,
data=false}]
[2012-09-26 13:11:12,753][WARN ][discovery.zen ] [Sinclar,
Nekra] received a join request for an existing node
[[Nighthawk][dZwzxHghQSir5vESBs-InQ][inet[/10.10.16.4:9301]]{client=true,
data=false}]
[2012-09-26 13:11:15,757][WARN ][discovery.zen ] [Sinclar,
Nekra] received a join request for an existing node
[[Nighthawk][dZwzxHghQSir5vESBs-InQ][inet[/10.10.16.4:9301]]{client=true,
data=false}]
... (this repeats many times)
[2012-09-26 13:12:36,141][INFO ][cluster.service ] [Sinclar,
Nekra] removed
{[Nighthawk][dZwzxHghQSir5vESBs-InQ][inet[/10.10.16.4:9301]]{client=true,
data=false},}, reason:
zen-disco-node_failed([Nighthawk][dZwzxHghQSir5vESBs-InQ][inet[/10.10.16.4:9301]]{client=true,
data=false}), reason transport disconnected (with verified connect)

Grails application throws this exception:

Caused by: org.elasticsearch.discovery.MasterNotDiscoveredException: waited

for [30s]
at
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$3.onTimeout(TransportMasterNodeOperationAction.java:169)
at
org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:359)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

What might be causing this behavior? Why the client node is not connecting
to the master?

Thank you,

Michael

--

2

Hmmm. Do you have some settings that override defaults (cluster.name,
unicast, …)?

If you have launched your ES server (standalone) with another cluster name,
that’s the reason.

David.

De : elasticsearch@googlegroups.com [mailto:elasticsearch@googlegroups.com]
De la part de Michael Smolyak
Envoyé : mercredi 26 septembre 2012 20:06
À : elasticsearch@googlegroups.com
Objet : Unable to create a Client node

I am trying to use Java API to create an ES Client. My Grails worked fine
utilizing ElasticSearch in the local (same JVM) mode. For clustering
purposes I am trying to change the application to use the "node" mode. The
application uses Grail ElasticSearch plugin. I first start ES server (I am
using 0.19.3 on both ends). After the server starts I start the Grails
application which through the plugin runs the following code:

def nb = NodeBuilder.nodeBuilder()

nb.client(true)

node = nb.node()

node.start()

def client = node.client()

...

The Grails application is unable to start because the new node does not
connect to the server. I can see the node trying to connect.

Here is the server log:

[2012-09-26 13:10:34,023][INFO ][node ] [Sinclar, Nekra]
{0.19.3}[9146]: initializing ...
[2012-09-26 13:10:34,027][INFO ][plugins ] [Sinclar, Nekra]
loaded [], sites []
[2012-09-26 13:10:35,519][INFO ][node ] [Sinclar, Nekra]
{0.19.3}[9146]: initialized
[2012-09-26 13:10:35,519][INFO ][node ] [Sinclar, Nekra]
{0.19.3}[9146]: starting ...
[2012-09-26 13:10:35,570][INFO ][transport ] [Sinclar, Nekra]
bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address
{inet[/10.10.16.4:9300]}
[2012-09-26 13:10:38,608][INFO ][cluster.service ] [Sinclar, Nekra]
new_master [Sinclar, Nekra][un4UD12IRg2S5aGPoOc-rQ][inet[/10.10.16.4:9300]],
reason: zen-disco-join (elected_as_master)
[2012-09-26 13:10:38,643][INFO ][discovery ] [Sinclar, Nekra]
elasticsearch/un4UD12IRg2S5aGPoOc-rQ
[2012-09-26 13:10:38,652][INFO ][http ] [Sinclar, Nekra]
bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address
{inet[/10.10.16.4:9200]}
[2012-09-26 13:10:38,652][INFO ][node ] [Sinclar, Nekra]
{0.19.3}[9146]: started
[2012-09-26 13:10:38,653][INFO ][gateway ] [Sinclar, Nekra]
recovered [0] indices into cluster_state
[2012-09-26 13:11:06,737][INFO ][cluster.service ] [Sinclar, Nekra]
added
{[Nighthawk][dZwzxHghQSir5vESBs-InQ][inet[/10.10.16.4:9301]]{client=true,
data=false},}, reason: zen-disco-receive(join from
node[[Nighthawk][dZwzxHghQSir5vESBs-InQ][inet[/10.10.16.4:9301]]{client=true
, data=false}])
[2012-09-26 13:11:09,748][WARN ][discovery.zen ] [Sinclar, Nekra]
received a join request for an existing node
[[Nighthawk][dZwzxHghQSir5vESBs-InQ][inet[/10.10.16.4:9301]]{client=true,
data=false}]
[2012-09-26 13:11:12,753][WARN ][discovery.zen ] [Sinclar, Nekra]
received a join request for an existing node
[[Nighthawk][dZwzxHghQSir5vESBs-InQ][inet[/10.10.16.4:9301]]{client=true,
data=false}]
[2012-09-26 13:11:15,757][WARN ][discovery.zen ] [Sinclar, Nekra]
received a join request for an existing node
[[Nighthawk][dZwzxHghQSir5vESBs-InQ][inet[/10.10.16.4:9301]]{client=true,
data=false}]
... (this repeats many times)
[2012-09-26 13:12:36,141][INFO ][cluster.service ] [Sinclar, Nekra]
removed
{[Nighthawk][dZwzxHghQSir5vESBs-InQ][inet[/10.10.16.4:9301]]{client=true,
data=false},}, reason:
zen-disco-node_failed([Nighthawk][dZwzxHghQSir5vESBs-InQ][inet[/10.10.16.4:9
301]]{client=true, data=false}), reason transport disconnected (with
verified connect)

Grails application throws this exception:

Caused by: org.elasticsearch.discovery.MasterNotDiscoveredException: waited
for [30s]
at
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$3
.onTimeout(TransportMasterNodeOperationAction.java:169)
at
org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(I
nternalClusterService.java:359)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.ja
va:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:9
08)
at java.lang.Thread.run(Thread.java:662)

What might be causing this behavior? Why the client node is not connecting
to the master?

Thank you,

Michael

--

--

3

David,

I simply downloaded the binary distribution of ES and ran

bin/elasticsearch -f

On the client side I execute the code quoted in my original post. Thus
neither the server nor the client has any custom configuration.

Should they? Apparently the client is trying to connect and the server sees
it. Why doesn't the client succeed? What can I do to debug this issue?

Thank you,

Michael

On Wednesday, September 26, 2012 3:14:16 PM UTC-4, David Pilato wrote:

Hmmm. Do you have some settings that override defaults (cluster.name,
unicast, …)?

If you have launched your ES server (standalone) with another cluster
name, that’s the reason.

David.

--

4

So, you are using defaults.
It should work. I don't know if you have any "firewall" rules or something like
that which is blocking 9300 and 9301 ports???

When you launch only the standalone node, can you curl http://localhost:9200 ?
When you launch only the grail node, can you curl http://localhost:9200 ?

When you launch both nodes, can you curl http://localhost:9200 and curl
http://localhost:9201 ?

David

Le 26 septembre 2012 à 22:04, Michael Smolyak msmolyak.nextcentury@gmail.com a
écrit :

David,

I simply downloaded the binary distribution of ES and ran

bin/elasticsearch -f

On the client side I execute the code quoted in my original post. Thus
neither the server nor the client has any custom configuration.

Should they? Apparently the client is trying to connect and the server sees
it. Why doesn't the client succeed? What can I do to debug this issue?

Thank you,

Michael

On Wednesday, September 26, 2012 3:14:16 PM UTC-4, David Pilato wrote:

Hmmm. Do you have some settings that override defaults (cluster.name
http://cluster.name , unicast, …)?

If you have launched your ES server (standalone) with another cluster
name, that’s the reason.

David.

--

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--

5

It appears the problem was in the conflict between the version of
elasticseach used on the client and on the server. After I synchronized he
two version I was able to connect.

Thank you for your help, David.

On Wednesday, September 26, 2012 5:06:29 PM UTC-4, David Pilato wrote:

So, you are using defaults.
It should work. I don't know if you have any "firewall" rules or
something like that which is blocking 9300 and 9301 ports???

When you launch only the standalone node, can you curl
http://localhost:9200 ?
When you launch only the grail node, can you curl http://localhost:9200?

When you launch both nodes, can you curl http://localhost:9200 and curl
http://localhost:9201 ?

David

Le 26 septembre 2012 à 22:04, Michael Smolyak <msmolyak.n...@gmail.com<javascript:>>
a écrit :

David,

I simply downloaded the binary distribution of ES and ran

bin/elasticsearch -f

On the client side I execute the code quoted in my original post. Thus
neither the server nor the client has any custom configuration.

Should they? Apparently the client is trying to connect and the server
sees it. Why doesn't the client succeed? What can I do to debug this issue?

Thank you,

Michael

On Wednesday, September 26, 2012 3:14:16 PM UTC-4, David Pilato wrote:

Hmmm. Do you have some settings that override defaults (cluster.name,
unicast, …)?

If you have launched your ES server (standalone) with another cluster
name, that’s the reason.

David.

--

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--