Error as an when data node is trying to connect to master node in elasticsearch clustering

Elastic Stack Elasticsearch
1

Getting below error as an when data node is trying to connect to master node in elasticsearch clustering

[node ] [data1] stopped
[2018-09-24 09:11:55,728][INFO ][node ] [data1] closing ...
[2018-09-24 09:11:55,737][INFO ][node ] [data1] closed
[2018-09-24 09:11:56,803][INFO ][node ] [data1] version[2.3.1], pid[5076], build[bd98092/2016-04-04T12:25:05Z]
[2018-09-24 09:11:56,810][INFO ][node ] [data1] initializing ...
[2018-09-24 09:11:57,556][INFO ][plugins ] [data1] modules [reindex, lang-expression, lang-groovy], plugins [], sites []
[2018-09-24 09:11:57,580][INFO ][env ] [data1] using [1] data paths, mounts [[/ (/dev/xvda1)]], net usable_space [27.1gb]$
[2018-09-24 09:11:57,580][INFO ][env ] [data1] heap size [1015.6mb], compressed ordinary object pointers [true]
[2018-09-24 09:11:57,581][WARN ][env ] [data1] max file descriptors [65535] for elasticsearch process likely too low, con$
[2018-09-24 09:12:00,464][INFO ][node ] [data1] initialized
[2018-09-24 09:12:00,468][INFO ][node ] [data1] starting ...
[2018-09-24 09:12:00,620][INFO ][transport ] [data1] publish_address {172.31.0.169:9300}, bound_addresses {[::]:9300}
[2018-09-24 09:12:00,633][INFO ][discovery ] [data1] elasticsearch/GcBsp-0xQjaZb2TYBrp4nA
[2018-09-24 09:12:30,645][WARN ][discovery ] [data1] waited for 30s and no initial state was set by the discovery
[2018-09-24 09:12:30,652][INFO ][http ] [data1] publish_address {172.31.0.169:9200}, bound_addresses {[::]:9200}
[2018-09-24 09:12:30,652][INFO ][node ] [data1] started
[2018-09-24 09:12:33,724][WARN ][discovery.zen ] [data1] failed to connect to master [{master}{Uc3ukg4DQWi7-oaU0RvnIg}{172.31.20.15$
ConnectTransportException[[master][172.31.20.157:9300] connect_timeout[30s]]; nested: ConnectTimeoutException[connection timed out: /172.31.20$
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:987)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:920)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:893)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:260)
at org.elasticsearch.discovery.zen.ZenDiscovery.joinElectedMaster(ZenDiscovery.java:434)
at org.elasticsearch.discovery.zen.ZenDiscovery.innerJoinCluster(ZenDiscovery.java:386)
at org.elasticsearch.discovery.zen.ZenDiscovery.access$4800(ZenDiscovery.java:91)
at org.elasticsearch.discovery.zen.ZenDiscovery$JoinThreadControl$1.run(ZenDiscovery.java:1237)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.jboss.netty.channel.ConnectTimeoutException: connection timed out: /172.31.20.157:9300
at org.jboss.netty.channel.socket.nio.NioClientBoss.processConnectTimeout(NioClientBoss.java:139)
at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:83)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)

2

Hi Cheshtha,

Are you sure those two nodes are able to communicate. In particular, is 172.31.20.157 reachable on port 9300? Might there be any firewalls blocking access between the machines? You could try to use telnet to see if you can actually connect to that port on the machine.

3

hey fkelbert ,

Yes , i have tried telnet . I am able to connect to port 9300 between machines and viceversa

4

I have changed port from 9300 to 9400 and now getting below response :

It should display 2 nodes .. but unfortunately it is only showing single node
on hitting http://34.254.71.33:9200/_cluster/state?pretty

{
  "cluster_name" : "elasticsearch",
  "version" : 2,
  "state_uuid" : "6bHwgDoyRs6zAUOaVV4Epg",
  "master_node" : "BYDgki34R5GDssW2PYj85w",
  "blocks" : { },
  "nodes" : {
    "BYDgki34R5GDssW2PYj85w" : {
      "name" : "master",
      "transport_address" : "172.31.20.157:9300",
      "attributes" : {
        "data" : "false",
        "master" : "true"
      }
    }
  },
  "metadata" : {
    "cluster_uuid" : "VaSs84CFSzq4odeG1TyZIQ",
    "templates" : { },
    "indices" : { }
  },
  "routing_table" : {
    "indices" : { }
  },
  "routing_nodes" : {
    "unassigned" : [ ],
    "nodes" : { }
  }
}
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.