Seperate interfaces for clustering and querying


(Jim Mitchener) #1

I am trying to setup an elastic cluster with a different network.bind_host
and network.publish_host interfaces. I cannot for the life of me get this
to work with 2 real interfaces. If I set both to the same interface or one
to loopback and the other to a real interface it works. However, if I try
to set publish_host and bind_host to different interfaces I get errors on
the publish_host timing out trying to reconnect to itself.

Am I not understanding how publish_host is supposed to work? Should it not
be set to a local IP that I wish to talk to the cluster on? I can post full
trace logs if needed.

Thanks,
Jim


(Shay Banon) #2

The publish host will be the address used to broadcast to the other nodes
so they will use it to connect to the cluster with.

On Fri, May 18, 2012 at 3:58 AM, Jim Mitchener jcm@packetpan.org wrote:

I am trying to setup an elastic cluster with a different network.bind_host
and network.publish_host interfaces. I cannot for the life of me get this
to work with 2 real interfaces. If I set both to the same interface or one
to loopback and the other to a real interface it works. However, if I try
to set publish_host and bind_host to different interfaces I get errors on
the publish_host timing out trying to reconnect to itself.

Am I not understanding how publish_host is supposed to work? Should it not
be set to a local IP that I wish to talk to the cluster on? I can post full
trace logs if needed.

Thanks,
Jim


(Shay Banon) #3

Sorry, to connect to the node with.

On Sun, May 20, 2012 at 10:13 PM, Shay Banon kimchy@gmail.com wrote:

The publish host will be the address used to broadcast to the other nodes
so they will use it to connect to the cluster with.

On Fri, May 18, 2012 at 3:58 AM, Jim Mitchener jcm@packetpan.org wrote:

I am trying to setup an elastic cluster with a different
network.bind_host and network.publish_host interfaces. I cannot for the
life of me get this to work with 2 real interfaces. If I set both to the
same interface or one to loopback and the other to a real interface it
works. However, if I try to set publish_host and bind_host to different
interfaces I get errors on the publish_host timing out trying to reconnect
to itself.

Am I not understanding how publish_host is supposed to work? Should it
not be set to a local IP that I wish to talk to the cluster on? I can post
full trace logs if needed.

Thanks,
Jim


(swendel) #4

hey guys,

im also trying to get seperate interfaces configuration for the cluster and http communication running, but no luke.

vagrant@ubuntu10044:~$ sudo netstat -tlpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.1:32000 0.0.0.0:* LISTEN 5655/java
tcp6 0 0 10.0.2.15:9300 :::* LISTEN 5655/java
tcp6 0 0 10.0.2.15:9200 :::* LISTEN 5655/java

vi /etc/elasticsearch/elasticsearch.yml
############################## Network And HTTP ###############################

network.bind_host: eth0:ipv4
network.publish_host: eth1:ipv4

################################### Gateway ###################################

less /var/log/elasticsearch/elasticsearch.log
[2012-06-08 00:00:26,884][WARN ][cluster.service ] [ubuntu10044] failed to reconnect to node [ubuntu10044][WHZxwr9VQeyVpASDgHSJ6w][inet[/192.168.0.4:9300]]
org.elasticsearch.transport.ConnectTransportException: [ubuntu10044][inet[/192.168.0.4:9300]] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:560)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:503)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:482)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:128)
at org.elasticsearch.cluster.service.InternalClusterService$ReconnectToNodes.run(InternalClusterService.java:377)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:592)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:400)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:362)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:284)
at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:102)

You can see how i tried to deploy elasticsearch in my chef cookbook at git hub:
https://github.com/sebwendel/chef-elasticsearch/tree/0.1.0

do you have any ideas?

Regards,
Sebastian


(Shay Banon) #5

If you want separate interfaces for internal cluster (transport)
communication and http, the settings should be transport.host for the inner
cluster communication, and http.host for the http one.

On Fri, Jun 8, 2012 at 1:56 PM, swendel s.wendel@sourceindex.de wrote:

hey guys,

im also trying to get seperate interfaces configuration for the cluster and
http communication running, but no luke.

vagrant@ubuntu10044:~$ sudo netstat -tlpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
PID/Program name
tcp 0 0 127.0.0.1:32000 0.0.0.0:* LISTEN
5655/java
tcp6 0 0 10.0.2.15:9300 :::* LISTEN
5655/java
tcp6 0 0 10.0.2.15:9200 :::* LISTEN
5655/java

vi /etc/elasticsearch/elasticsearch.yml
############################## Network And HTTP
###############################

network.bind_host: eth0:ipv4
network.publish_host: eth1:ipv4

################################### Gateway
###################################

less /var/log/elasticsearch/elasticsearch.log
[2012-06-08 00:00:26,884][WARN ][cluster.service ] [ubuntu10044]
failed to reconnect to node
[ubuntu10044][WHZxwr9VQeyVpASDgHSJ6w][inet[/192.168.0.4:9300]]
org.elasticsearch.transport.ConnectTransportException:
[ubuntu10044][inet[/192.168.0.4:9300]] connect_timeout[30s]
at

org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:560)
at

org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:503)
at

org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:482)
at

org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:128)
at

org.elasticsearch.cluster.service.InternalClusterService$ReconnectToNodes.run(InternalClusterService.java:377)
at

java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:592)
at

org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:400)
at

org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:362)
at

org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:284)
at

org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:102)

You can see how i tried to deploy elasticsearch in my chef cookbook at git
hub:
https://github.com/sebwendel/chef-elasticsearch/tree/0.1.0

do you have any ideas?

Regards,
Sebastian

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Seperate-interfaces-for-clustering-and-querying-tp4000112p4018970.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.


(swendel) #6

Thanks for the answer.

But whare ist the difference to: network.bind_host and network.publish_host?

And how comes that the documentation tells something completely different?

http://www.elasticsearch.org/guide/reference/modules/network.html

Is this a complete new implementation? Or did i misunderstood the
documentation?

greetings,
Sebastian

2012/6/11 Shay Banon kimchy@gmail.com

If you want separate interfaces for internal cluster (transport)
communication and http, the settings should be transport.host for the inner
cluster communication, and http.host for the http one.

On Fri, Jun 8, 2012 at 1:56 PM, swendel s.wendel@sourceindex.de wrote:

hey guys,

im also trying to get seperate interfaces configuration for the cluster
and
http communication running, but no luke.

vagrant@ubuntu10044:~$ sudo netstat -tlpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
PID/Program name
tcp 0 0 127.0.0.1:32000 0.0.0.0:*
LISTEN
5655/java
tcp6 0 0 10.0.2.15:9300 :::*
LISTEN
5655/java
tcp6 0 0 10.0.2.15:9200 :::*
LISTEN
5655/java

vi /etc/elasticsearch/elasticsearch.yml
############################## Network And HTTP
###############################

network.bind_host: eth0:ipv4
network.publish_host: eth1:ipv4

################################### Gateway
###################################

less /var/log/elasticsearch/elasticsearch.log
[2012-06-08 00:00:26,884][WARN ][cluster.service ] [ubuntu10044]
failed to reconnect to node
[ubuntu10044][WHZxwr9VQeyVpASDgHSJ6w][inet[/192.168.0.4:9300]]
org.elasticsearch.transport.ConnectTransportException:
[ubuntu10044][inet[/192.168.0.4:9300]] connect_timeout[30s]
at

org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:560)
at

org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:503)
at

org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:482)
at

org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:128)
at

org.elasticsearch.cluster.service.InternalClusterService$ReconnectToNodes.run(InternalClusterService.java:377)
at

java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:592)
at

org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:400)
at

org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:362)
at

org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:284)
at

org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:102)

You can see how i tried to deploy elasticsearch in my chef cookbook at git
hub:
https://github.com/sebwendel/chef-elasticsearch/tree/0.1.0

do you have any ideas?

Regards,
Sebastian

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Seperate-interfaces-for-clustering-and-querying-tp4000112p4018970.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.


(swendel) #7

hi Shay,

your advice worked as i hoped, thanks.

vagrant@ubuntu1204:~$ cat /etc/elasticsearch/elasticsearch.ym
http.host: eth0:ipv4
transport.host: eth1:ipv4

but :wink: now there is another problem.

vagrant@ubuntu1204:~$ sudo netstat -tulpn | grep java
tcp 0 0 127.0.0.1:32000 0.0.0.0:* LISTEN
8184/java
tcp6 0 0 10.0.2.15:9200 :::* LISTEN
8184/java
tcp6 0 0 192.168.0.5:9300 :::* LISTEN
8184/java
udp6 0 0 :::54328 :::*
8184/java

Zen Discovery is not listening on the same interface that the transport.

If there any way to also control that? didn't found any hint in the
documentation or in the net.

2012/6/11 Sebastian Wendel s.wendel@sourceindex.de

Thanks for the answer.

But whare ist the difference to: network.bind_host
and network.publish_host?

And how comes that the documentation tells something completely different?

http://www.elasticsearch.org/guide/reference/modules/network.html

Is this a complete new implementation? Or did i misunderstood the
documentation?

greetings,
Sebastian

2012/6/11 Shay Banon kimchy@gmail.com

If you want separate interfaces for internal cluster (transport)
communication and http, the settings should be transport.host for the inner
cluster communication, and http.host for the http one.

On Fri, Jun 8, 2012 at 1:56 PM, swendel s.wendel@sourceindex.de wrote:

hey guys,

im also trying to get seperate interfaces configuration for the cluster
and
http communication running, but no luke.

vagrant@ubuntu10044:~$ sudo netstat -tlpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
PID/Program name
tcp 0 0 127.0.0.1:32000 0.0.0.0:*
LISTEN
5655/java
tcp6 0 0 10.0.2.15:9300 :::*
LISTEN
5655/java
tcp6 0 0 10.0.2.15:9200 :::*
LISTEN
5655/java

vi /etc/elasticsearch/elasticsearch.yml
############################## Network And HTTP
###############################

network.bind_host: eth0:ipv4
network.publish_host: eth1:ipv4

################################### Gateway
###################################

less /var/log/elasticsearch/elasticsearch.log
[2012-06-08 00:00:26,884][WARN ][cluster.service ] [ubuntu10044]
failed to reconnect to node
[ubuntu10044][WHZxwr9VQeyVpASDgHSJ6w][inet[/192.168.0.4:9300]]
org.elasticsearch.transport.ConnectTransportException:
[ubuntu10044][inet[/192.168.0.4:9300]] connect_timeout[30s]
at

org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:560)
at

org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:503)
at

org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:482)
at

org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:128)
at

org.elasticsearch.cluster.service.InternalClusterService$ReconnectToNodes.run(InternalClusterService.java:377)
at

java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:592)
at

org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:400)
at

org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:362)
at

org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:284)
at

org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:102)

You can see how i tried to deploy elasticsearch in my chef cookbook at
git
hub:
https://github.com/sebwendel/chef-elasticsearch/tree/0.1.0

do you have any ideas?

Regards,
Sebastian

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Seperate-interfaces-for-clustering-and-querying-tp4000112p4018970.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.

--
Mit freundlichen Grüßen,
Sebastian Wendel.

//////////////////////////////////////////////////////////////////////////////////////

SourceIndex IT-Services
Wohlwillstraße 2, 20359 Hamburg

Office: +49(0)40-209 344 900
Fax: +49(0)40-209 344 909
Email: s.wendel@sourceindex.de

Web: http://www.sourceindex.de
Xing: http://www.xing.com/profile/sebastian_wendel

WARNING / LEGAL TEXT: This message is intended only for the use of the
individual or entity to which it is addressed and may contain
information which is privileged, confidential, proprietary, or exempt
from disclosure under applicable law. If you are not the intended
recipient or the person responsible for delivering the message to the
intended recipient, you are strictly prohibited from disclosing,
distributing, copying, or in any way using this message. If you have
received this communication in error, please notify the sender and
destroy and delete any copies you may have received.


(Shay Banon) #8

The network.host setting simply sets it to all sockets based modules (http
and transport).

On Mon, Jun 11, 2012 at 8:31 PM, Sebastian Wendel
s.wendel@sourceindex.dewrote:

Thanks for the answer.

But whare ist the difference to: network.bind_host
and network.publish_host?

And how comes that the documentation tells something completely different?

http://www.elasticsearch.org/guide/reference/modules/network.html

Is this a complete new implementation? Or did i misunderstood the
documentation?

greetings,
Sebastian

2012/6/11 Shay Banon kimchy@gmail.com

If you want separate interfaces for internal cluster (transport)
communication and http, the settings should be transport.host for the inner
cluster communication, and http.host for the http one.

On Fri, Jun 8, 2012 at 1:56 PM, swendel s.wendel@sourceindex.de wrote:

hey guys,

im also trying to get seperate interfaces configuration for the cluster
and
http communication running, but no luke.

vagrant@ubuntu10044:~$ sudo netstat -tlpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
PID/Program name
tcp 0 0 127.0.0.1:32000 0.0.0.0:*
LISTEN
5655/java
tcp6 0 0 10.0.2.15:9300 :::*
LISTEN
5655/java
tcp6 0 0 10.0.2.15:9200 :::*
LISTEN
5655/java

vi /etc/elasticsearch/elasticsearch.yml
############################## Network And HTTP
###############################

network.bind_host: eth0:ipv4
network.publish_host: eth1:ipv4

################################### Gateway
###################################

less /var/log/elasticsearch/elasticsearch.log
[2012-06-08 00:00:26,884][WARN ][cluster.service ] [ubuntu10044]
failed to reconnect to node
[ubuntu10044][WHZxwr9VQeyVpASDgHSJ6w][inet[/192.168.0.4:9300]]
org.elasticsearch.transport.ConnectTransportException:
[ubuntu10044][inet[/192.168.0.4:9300]] connect_timeout[30s]
at

org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:560)
at

org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:503)
at

org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:482)
at

org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:128)
at

org.elasticsearch.cluster.service.InternalClusterService$ReconnectToNodes.run(InternalClusterService.java:377)
at

java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:592)
at

org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:400)
at

org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:362)
at

org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:284)
at

org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:102)

You can see how i tried to deploy elasticsearch in my chef cookbook at
git
hub:
https://github.com/sebwendel/chef-elasticsearch/tree/0.1.0

do you have any ideas?

Regards,
Sebastian

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Seperate-interfaces-for-clustering-and-querying-tp4000112p4018970.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.


(system) #9