Elastic Search Cluster behind Loadbalancer


(PKD) #1

Hi,

I have 3 Elastic Search Nodes (1.0.2) setup which are behind a
loadbalancer. We have some code written which uses transport client to
connect to independent nodes and populate data but the same code when we
point to loadbalancer and provide the cluster name it throws an error :
org.elasticsearch.client.transport.NoNodeAvailableException: No node
available

Please let me know is there any different mechanism to connect to Elastic
Search through load balancer ?

Thanks in advance.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/149537ea-e766-44c5-84b1-a42c9a15c98d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(PKD) #2

The code that we are using is :

Settings settings = ImmutableSettings.settingsBuilder()
.put("cluster.name", "dev").put( "client.transport.sniff",
true ) .build();
Client client = new TransportClient(settings)

    .addTransportAddress(new 

InetSocketTransportAddress("", 9300));

On Thursday, March 27, 2014 10:26:14 PM UTC+5:30, PKD wrote:

Hi,

I have 3 Elastic Search Nodes (1.0.2) setup which are behind a
loadbalancer. We have some code written which uses transport client to
connect to independent nodes and populate data but the same code when we
point to loadbalancer and provide the cluster name it throws an error :
org.elasticsearch.client.transport.NoNodeAvailableException: No node
available

Please let me know is there any different mechanism to connect to Elastic
Search through load balancer ?

Thanks in advance.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0c7a0e0b-541d-443f-80dd-ad8f62f1814a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Binh Ly-2) #3

I am curious which LB are you using? And also I'm assuming you have the
9300 port open and forwarded to your cluster?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8538918c-2370-45f9-bab0-e9de0225632a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(PKD) #4

From the Logs it seems that the ports are open and the java code is able to
connect but it gets disconnected immediately.

[2014-03-28 14:20:24,604][DEBUG][org.elasticsearch.client.transport]
[Balder] node_sampler_interval[5s]
[2014-03-28
14:20:24,633][DEBUG][org.elasticsearch.netty.channel.socket.nio.SelectorUtil]
Using select timeout of 500
[2014-03-28
14:20:24,633][DEBUG][org.elasticsearch.netty.channel.socket.nio.SelectorUtil]
Epoll-bug workaround enabled = false
[2014-03-28 14:20:24,687][DEBUG][org.elasticsearch.client.transport]
[Balder] adding address [[#transport#-1][kd-win7][inet[/1.1.0.160:9300]]]
[2014-03-28 14:20:24,956][DEBUG][org.elasticsearch.transport.netty]
[Balder] connected to node [[#transport#-1][kd-win7][inet[/1.1.0.160:9300]]]
[2014-03-28 14:20:26,068][DEBUG][org.elasticsearch.transport.netty]
[Balder] connected to node
[[db0-021][J7_LlmwlQ2SChW9_bcFW2g][deb0-021][inet[/10.1.0.160:9300]]]
[2014-03-28 14:20:30,775][DEBUG][org.elasticsearch.transport.netty]
[Balder] disconnected from
[[db0-021][J7_LlmwlQ2SChW9_bcFW2g][deb0-021][inet[/10.1.0.160:9300]]],
channel closed event
Exception in thread "main"
org.elasticsearch.client.transport.NoNodeAvailableException: No node
available
at
org.elasticsearch.client.transport.TransportClientNodesService$RetryListener.onFailure(TransportClientNodesService.java:263)
at
org.elasticsearch.action.TransportActionNodeProxy$1.handleException(TransportActionNodeProxy.java:89)
at
org.elasticsearch.transport.TransportService$2.run(TransportService.java:206)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

On Friday, March 28, 2014 12:01:44 AM UTC+5:30, Binh Ly wrote:

I am curious which LB are you using? And also I'm assuming you have the
9300 port open and forwarded to your cluster?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0959a766-1f8b-40e6-8cda-929096ed33ca%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(PKD) #5

[2014-03-28 14:20:24,604][DEBUG][org.
elasticsearch.client.transport] [Balder] node_sampler_interval[5s]
[2014-03-28
14:20:24,633][DEBUG][org.elasticsearch.netty.channel.socket.nio.SelectorUtil]
Using select timeout of 500
[2014-03-28
14:20:24,633][DEBUG][org.elasticsearch.netty.channel.socket.nio.SelectorUtil]
Epoll-bug workaround enabled = false
[2014-03-28 14:20:24,687][DEBUG][org.elasticsearch.client.transport]
[Balder] adding address [[#transport#-1][kd-win7][inet[/1.1.0.16:9300]]]
[2014-03-28 14:20:24,956][DEBUG][org.elasticsearch.transport.netty]
[Balder] connected to node [[#transport#-1][kd-win7][inet[/1.1.0.16:9300]]]
[2014-03-28 14:20:26,068][DEBUG][org.elasticsearch.transport.netty]
[Balder] connected to node
[[db0-021][J7_LlmwlQ2SChW9_bcFW2g][deb0-021][inet[/1.1.0.16:9300]]]
[2014-03-28 14:20:30,775][DEBUG][org.elasticsearch.transport.netty]
[Balder] disconnected from
[[db0-021][J7_LlmwlQ2SChW9_bcFW2g][deb0-021][inet[/1.1.0.16:9300]]],
channel closed event
Exception in thread "main"
org.elasticsearch.client.transport.NoNodeAvailableException: No node
available
at
org.elasticsearch.client.transport.TransportClientNodesService$RetryListener.onFailure(TransportClientNodesService.java:263)
at
org.elasticsearch.action.TransportActionNodeProxy$1.handleException(TransportActionNodeProxy.java:89)
at
org.elasticsearch.transport.TransportService$2.run(TransportService.java:206)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

On Friday, March 28, 2014 12:01:44 AM UTC+5:30, Binh Ly wrote:

I am curious which LB are you using? And also I'm assuming you have the
9300 port open and forwarded to your cluster?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/aa99e615-e974-46dd-80da-b1957e86c8bf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(PKD) #6

From the java client logs it seems that it able to connect to elastic
search node through load balancer.

[2014-03-28 14:20:24,604][DEBUG][org.
elasticsearch.client.transport] [Balder] node_sampler_interval[5s]
[2014-03-28
14:20:24,633][DEBUG][org.elasticsearch.netty.channel.socket.nio.SelectorUtil]
Using select timeout of 500
[2014-03-28
14:20:24,633][DEBUG][org.elasticsearch.netty.channel.socket.nio.SelectorUtil]
Epoll-bug workaround enabled = false
[2014-03-28 14:20:24,687][DEBUG][org.elasticsearch.client.transport]
[Balder] adding address [[#transport#-1][kd-win7][inet[/1.1.0.16:9300]]]
[2014-03-28 14:20:24,956][DEBUG][org.elasticsearch.transport.netty]
[Balder] connected to node [[#transport#-1][kd-win7][inet[/1.1.0.16:9300]]]
[2014-03-28 14:20:26,068][DEBUG][org.elasticsearch.transport.netty]
[Balder] connected to node
[[db0-021][J7_LlmwlQ2SChW9_bcFW2g][deb0-021][inet[/1.1.0.16:9300]]]
[2014-03-28 14:20:30,775][DEBUG][org.elasticsearch.transport.netty]
[Balder] disconnected from
[[db0-021][J7_LlmwlQ2SChW9_bcFW2g][deb0-021][inet[/1.1.0.16:9300]]],
channel closed event
Exception in thread "main"
org.elasticsearch.client.transport.NoNodeAvailableException: No node
available
at
org.elasticsearch.client.transport.TransportClientNodesService$RetryListener.onFailure(TransportClientNodesService.java:263)
at
org.elasticsearch.action.TransportActionNodeProxy$1.handleException(TransportActionNodeProxy.java:89)
at
org.elasticsearch.transport.TransportService$2.run(TransportService.java:206)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

On Friday, March 28, 2014 12:01:44 AM UTC+5:30, Binh Ly wrote:

I am curious which LB are you using? And also I'm assuming you have the
9300 port open and forwarded to your cluster?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/59444918-d25d-422d-b709-ce6f77581882%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(PKD) #7

From the java client logs it seems that it able to connect to elastic
search node through load balancer (1.1.0.16).

[2014-03-28 14:20:24,604][DEBUG][org.
elasticsearch.client.transport] [Balder] node_sampler_interval[5s]
[2014-03-28
14:20:24,633][DEBUG][org.elasticsearch.netty.channel.socket.nio.SelectorUtil]
Using select timeout of 500
[2014-03-28
14:20:24,633][DEBUG][org.elasticsearch.netty.channel.socket.nio.SelectorUtil]
Epoll-bug workaround enabled = false
[2014-03-28 14:20:24,687][DEBUG][org.elasticsearch.client.transport]
[Balder] adding address [[#transport#-1][kd-win7][inet[/1.1.0.16:9300]]]
[2014-03-28 14:20:24,956][DEBUG][org.elasticsearch.transport.netty]
[Balder] connected to node [[#transport#-1][kd-win7][inet[/1.1.0.16:9300]]]
[2014-03-28 14:20:26,068][DEBUG][org.elasticsearch.transport.netty]
[Balder] connected to node
[[db0-021][J7_LlmwlQ2SChW9_bcFW2g][deb0-021][inet[/1.1.0.16:9300]]]
[2014-03-28 14:20:30,775][DEBUG][org.elasticsearch.transport.netty]
[Balder] disconnected from
[[db0-021][J7_LlmwlQ2SChW9_bcFW2g][deb0-021][inet[/1.1.0.16:9300]]],
channel closed event
Exception in thread "main"
org.elasticsearch.client.transport.NoNodeAvailableException: No node
available
at
org.elasticsearch.client.transport.TransportClientNodesService$RetryListener.onFailure(TransportClientNodesService.java:263)
at
org.elasticsearch.action.TransportActionNodeProxy$1.handleException(TransportActionNodeProxy.java:89)
at
org.elasticsearch.transport.TransportService$2.run(TransportService.java:206)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

On Friday, March 28, 2014 12:01:44 AM UTC+5:30, Binh Ly wrote:

I am curious which LB are you using? And also I'm assuming you have the
9300 port open and forwarded to your cluster?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4e79564a-d711-48b1-ba06-071b9ad9e1c6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(PKD) #8

After Enabling client.transport.sniff parameter it is able to connect , but
it is connecting all of the nodes in the cluster. Ideally querying the load
balancer should return the response to which node to redirect will be taken
care by the load balancer. Is there any problem with the Java code or load
balancer settings ?

Please help.

[2014-03-28 15:40:16,822][DEBUG][org.elasticsearch.threadpool] [Mindmeld]

creating thread_pool [optimize], type [fixed], size [1], queue_size [null]
[2014-03-28 15:40:16,862][DEBUG][org.elasticsearch.transport.netty]
[Mindmeld] using worker_count[4], port[9300-9400], bind_host[null],
publish_host[null], compress[false], connect_timeout[30s],
connections_per_node[2/3/6/1/1], receive_predictor[512kb->512kb]
[2014-03-28 15:40:16,865][DEBUG][org.elasticsearch.client.transport]
[Mindmeld] node_sampler_interval[5s]
[2014-03-28
15:40:16,894][DEBUG][org.elasticsearch.netty.channel.socket.nio.SelectorUtil]
Using select timeout of 500
[2014-03-28
15:40:16,894][DEBUG][org.elasticsearch.netty.channel.socket.nio.SelectorUtil]
Epoll-bug workaround enabled = false
[2014-03-28 15:40:16,947][DEBUG][org.elasticsearch.client.transport]
[Mindmeld] adding address [[#transport#-1][kd-win7][inet[/1.1.0.160:9300]]]
[2014-03-28 15:40:17,208][DEBUG][org.elasticsearch.transport.netty]
[Mindmeld] connected to node
[[#transport#-1][kd-win7][inet[/1.1.0.160:9300]]]
[2014-03-28 15:40:17,765][DEBUG][org.elasticsearch.transport.netty]
[Mindmeld] connected to node
[[db0-121][zEUFq-q7ReWRCyNVd0Q5lA][db0-121][inet[/10.1.0.121:9300]]]
[2014-03-28 15:40:18,004][DEBUG][org.elasticsearch.transport.netty]
[Mindmeld] connected to node
[[db0-221][HsiqK4omR26vJuLgFHGCRA][db0-221][inet[/10.1.0.221:9300]]]
[2014-03-28 15:40:18,247][DEBUG][org.elasticsearch.transport.netty]
[Mindmeld] connected to node
[[db0-021][J7_LlmwlQ2SChW9_bcFW2g][db0-021][inet[/10.1.0.21:9300]]]

On Friday, March 28, 2014 3:37:28 PM UTC+5:30, PKD wrote:

From the java client logs it seems that it able to connect to elastic
search node through load balancer (1.1.0.16).

[2014-03-28 14:20:24,604][DEBUG][org.
elasticsearch.client.transport] [Balder] node_sampler_interval[5s]
[2014-03-28
14:20:24,633][DEBUG][org.elasticsearch.netty.channel.socket.nio.SelectorUtil]
Using select timeout of 500
[2014-03-28
14:20:24,633][DEBUG][org.elasticsearch.netty.channel.socket.nio.SelectorUtil]
Epoll-bug workaround enabled = false
[2014-03-28 14:20:24,687][DEBUG][org.elasticsearch.client.transport]
[Balder] adding address [[#transport#-1][kd-win7][inet[/1.1.0.16:9300]]]
[2014-03-28 14:20:24,956][DEBUG][org.elasticsearch.transport.netty]
[Balder] connected to node [[#transport#-1][kd-win7][inet[/1.1.0.16:9300]]]
[2014-03-28 14:20:26,068][DEBUG][org.elasticsearch.transport.netty]
[Balder] connected to node
[[db0-021][J7_LlmwlQ2SChW9_bcFW2g][deb0-021][inet[/1.1.0.16:9300]]]
[2014-03-28 14:20:30,775][DEBUG][org.elasticsearch.transport.netty]
[Balder] disconnected from
[[db0-021][J7_LlmwlQ2SChW9_bcFW2g][deb0-021][inet[/1.1.0.16:9300]]],
channel closed event
Exception in thread "main"
org.elasticsearch.client.transport.NoNodeAvailableException: No node
available
at
org.elasticsearch.client.transport.TransportClientNodesService$RetryListener.onFailure(TransportClientNodesService.java:263)
at
org.elasticsearch.action.TransportActionNodeProxy$1.handleException(TransportActionNodeProxy.java:89)
at
org.elasticsearch.transport.TransportService$2.run(TransportService.java:206)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

On Friday, March 28, 2014 12:01:44 AM UTC+5:30, Binh Ly wrote:

I am curious which LB are you using? And also I'm assuming you have the
9300 port open and forwarded to your cluster?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/219201e5-a0b3-4248-9876-0defb96dbd3e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #9