BindTransportException: Failed to bind to 9300-9400

krushnat_khawale · April 20, 2016, 11:31am

Trying to configure nodes(Network computers) under a cluster.

When I start first node, it starts successfully. Even second node starts successfully, but (second) terminates as soon as it is started with the following message,

[2016-04-20 15:27:04,877][WARN ][bootstrap                ] unable to install syscall filter: syscall filtering not supported for OS: 'Windows 7'
[2016-04-20 15:27:05,173][INFO ][node                     ] [node1] version[2.1.0], pid[8876], build[72cd1f1/2015-11-18T22:40:03Z]
[2016-04-20 15:27:05,173][INFO ][node                     ] [node1] initializing ...
[2016-04-20 15:27:05,235][INFO ][plugins                  ] [node1] loaded [], sites []
[2016-04-20 15:27:05,267][INFO ][env                      ] [node1] using [1] data paths, mounts [[Data (D:)]], net usable_space [147.6gb], net total_space [232.8gb], spins? [unknown], types [NTFS]
[2016-04-20 15:27:07,136][INFO ][node                     ] [node1] initialized
[2016-04-20 15:27:07,136][INFO ][node                     ] [node1] starting ...
[2016-04-20 15:27:07,214][INFO ][transport                ] [node1] publish_address {10.76.133.15:9301}, bound_addresses {10.76.133.15:9301}
[2016-04-20 15:27:07,214][INFO ][discovery                ] [node1] mycluster/qDX5__c9QEGa8uemmr_HVQ
[2016-04-20 15:27:10,271][INFO ][cluster.service          ] [node1] new_master {node1}{qDX5__c9QEGa8uemmr_HVQ}{10.76.133.15}{10.76.133.15:9301}, reason: zen-disco-join(elected_as_master, [0] joins received)
[2016-04-20 15:27:10,287][INFO ][http                     ] [node1] publish_address {10.76.133.15:9292}, bound_addresses {10.76.133.15:9292}
[2016-04-20 15:27:10,287][INFO ][node                     ] [node1] started
[2016-04-20 15:27:10,614][INFO ][gateway                  ] [node1] recovered [0] indices into cluster_state
[2016-04-20 15:30:07,099][INFO ][node                     ] [node1] stopping ...
[2016-04-20 15:30:07,123][INFO ][node                     ] [node1] stopped
[2016-04-20 15:30:07,123][INFO ][node                     ] [node1] closing ...
[2016-04-20 15:30:07,130][INFO ][node                     ] [node1] closed
[2016-04-20 15:31:14,414][WARN ][bootstrap                ] unable to install syscall filter: syscall filtering not supported for OS: 'Windows 7'
[2016-04-20 15:31:14,738][INFO ][node                     ] [node1] version[2.1.0], pid[11220], build[72cd1f1/2015-11-18T22:40:03Z]
[2016-04-20 15:31:14,738][INFO ][node                     ] [node1] initializing ...
[2016-04-20 15:31:14,811][INFO ][plugins                  ] [node1] loaded [], sites []
[2016-04-20 15:31:14,840][INFO ][env                      ] [node1] using [1] data paths, mounts [[Data (D:)]], net usable_space [147.6gb], net total_space [232.8gb], spins? [unknown], types [NTFS]
[2016-04-20 15:31:16,832][INFO ][node                     ] [node1] initialized
[2016-04-20 15:31:16,832][INFO ][node                     ] [node1] starting ...
[2016-04-20 15:31:16,956][ERROR][bootstrap                ] [node1] Exception
BindTransportException[Failed to bind to [9300-9400]]; nested: ChannelException[Failed to bind to: /10.76.170.72:9400]; nested: BindException[Cannot assign requested address: bind];
	at org.elasticsearch.transport.netty.NettyTransport.bindServerBootstrap(NettyTransport.java:474)
	at org.elasticsearch.transport.netty.NettyTransport.bindServerBootstrap(NettyTransport.java:442)
	at org.elasticsearch.transport.netty.NettyTransport.doStart(NettyTransport.java:319)
	at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:68)
	at org.elasticsearch.transport.TransportService.doStart(TransportService.java:170)
	at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:68)
	at org.elasticsearch.node.Node.start(Node.java:254)
	at org.elasticsearch.bootstrap.Bootstrap.start(Bootstrap.java:221)
	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:287)
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
Caused by: org.jboss.netty.channel.ChannelException: Failed to bind to: /10.76.170.72:9400
	at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272)
	at org.elasticsearch.transport.netty.NettyTransport$1.onPortNumber(NettyTransport.java:456)
	at org.elasticsearch.common.transport.PortsRange.iterate(PortsRange.java:69)
	at org.elasticsearch.transport.netty.NettyTransport.bindServerBootstrap(NettyTransport.java:452)
	... 9 more 
[2016-04-20 15:31:16,981][INFO ][node                     ] [node1] stopping ...
[2016-04-20 15:31:16,984][INFO ][node                     ] [node1] stopped
[2016-04-20 15:31:16,985][INFO ][node                     ] [node1] closing ...
[2016-04-20 15:31:16,990][INFO ][node                     ] [node1] closed

krushnat_khawale · April 20, 2016, 11:31am

Following is my configuration for both the nodes:

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please see the documentation for further information on configuration options:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/setup-configuration.html>
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
 cluster.name: mycluster
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
 node.name: node1
#
# Add custom attributes to the node:
#
# node.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
# path.data: /path/to/data
#
# Path to log files:
#
# path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
# bootstrap.mlockall: true
#
# Make sure that the `ES_HEAP_SIZE` environment variable is set to about half the memory
# available on the system and that the owner of the process is allowed to use this limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
 network.host: 10.76.133.15
#
# Set a custom port for HTTP:
#
 http.port: 9292
#
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html>
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
# gateway.recover_after_nodes: 3
#
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html>
#
# --------------------------------- Discovery ----------------------------------
#
# Elasticsearch nodes will find each other via unicast, by default.
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
 discovery.zen.ping.unicast.hosts: ["10.76.133.15:9292", "10.76.170.72:9292"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of nodes / 2 + 1):
#
# discovery.zen.minimum_master_nodes: 3
#
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery.html>
#
# ---------------------------------- Various -----------------------------------
#
# Disable starting multiple nodes on a single system:
#
# node.max_local_storage_nodes: 1
#
# Require explicit names when deleting indices:
#
# action.destructive_requires_name: true

magnusbaeck · April 24, 2016, 3:38pm

Caused by: org.jboss.netty.channel.ChannelException: Failed to bind to: /10.76.170.72:9400

Why would it try to bind to 10.76.170.72 when network.host is 10.76.133.15? Anyway, what's the machine's IP address?

Following is my configuration for both the nodes:

When you're setting network.host to a fixed IP address you can't use the same configuration file for both.

krushnat_khawale · April 25, 2016, 1:55pm

Yeah @magnusbaeck,

Keeping 10.75.133.15 as network.host in both the configurations is really a mistake.
I changed it to their own, and it worked.

But what is that list discovery.zen.ping.unicast.hosts ?
Do we need to to specify same list on both the machines or either of them if have it,
will it work?

magnusbaeck · April 25, 2016, 5:28pm

The unicast host list is used for each node to find the rest of the cluster. I suggest you list all cluster members or at least all master-eligible nodes. You can use the same list on all nodes. It's fine if a node lists its own address.

Cyril_Prince · February 1, 2017, 10:48am

Worked on my AWS Linux Machine

cluster.name: myES_Cluster
node.name: ESNODE_CYR
node.master: true
node.data: true
transport.host: localhost
transport.tcp.port: 9300
http.port: 9200
network.host: 0.0.0.0
discovery.zen.minimum_master_nodes: 2

I have tried with this on the elasticsearch.yml (key:value) and worked fine for me. But it takes 2 days to fix it , going on with ES Doc is so tough.

elasticH · February 9, 2017, 10:54am

Hello,
I have a VM (Sandbox HDP 2.5) and I installed elasticSearch and kibana on it.
I configured my elasticsearch.yml as you did, but still can't connect on my elasticsearch from my computer.
I tried with http://172.17.0.2:9200/ but still not working.

Do you any idea on how the network manage the communication between VmWare machine on linux?
Thanks

chenchu · May 16, 2017, 10:43am

@elasticH

I am Also Facing Same Issue Can You Please guide Me Bro

My .Ymal File

======================== Elasticsearch Configuration =========================

NOTE: Elasticsearch comes with reasonable defaults for most settings.

Before you set out to tweak and tune the configuration, make sure you

understand what are you trying to accomplish and the consequences.

The primary way of configuring a node is via this file. This template lists

the most important settings you may want to configure for a production cluster.

Please consult the documentation for further information on configuration options:

https://www.elastic.co/guide/en/elasticsearch/reference/index.html

---------------------------------- Cluster -----------------------------------

Use a descriptive name for your cluster:

#cluster.name: my-application
cluster.name: 'elasticSearch'

------------------------------------ Node ------------------------------------

Use a descriptive name for the node:

node.name : 'elasticSearch'

Add custom attributes to the node:

#node.attr.rack: r1

----------------------------------- Paths ------------------------------------

Path to directory where to store the data (separate multiple locations by comma):

#path.data: /path/to/data

Path to log files:

#path.logs: /path/to/logs

----------------------------------- Memory -----------------------------------

Lock the memory on startup:

#bootstrap.memory_lock: true

Make sure that the heap size is set to about half the memory available

on the system and that the owner of the process is allowed to use this

limit.

Elasticsearch performs poorly when the system is swapping the memory.

---------------------------------- Network -----------------------------------

Set the bind address to a specific IP (IPv4 or IPv6):

network.host : '52.66.132.152'

Set a custom port for HTTP:

http.port : 7200

For more information, consult the network module documentation.

--------------------------------- Discovery ----------------------------------

Pass an initial list of hosts to perform discovery when new node is started:

#The default list of hosts is ["127.0.0.1", "[::1]"]

discovery.zen.ping.unicast.hosts : ["52.66.132.152:7200"]

Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):