# Unable to add a new node **URL:** \ **Category:** Elasticsearch\ **Created:** [July 26, 2017, 6:23pm UTC](https://discuss.elastic.co/t/unable-to-add-a-new-node/94684 "2017-07-26T18:23:37Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Gkmgaurav](https://avatars.discourse-cdn.com/v4/letter/g/91b2a8/32.png) [@Gkmgaurav](https://discuss.elastic.co/u/Gkmgaurav)\ **Post date:** [July 26, 2017, 6:23pm UTC](https://discuss.elastic.co/t/unable-to-add-a-new-node/94684/1 "2017-07-26T18:23:37Z")
Hi friends, I am new to elastic search. It is possible someone might have asked this question previously. But, I am not able to find a proper solution. I had a 3 node cluster on AWS, with ver 1.5 . Out of the 3 nodes two died. And now I am left with only 1 master data node. I have now a totally new server, which i am trying to add as a data node. But it is not getting discovered and connecting to my master. How should I fix this? Please help. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [July 26, 2017, 7:56pm UTC](https://discuss.elastic.co/t/unable-to-add-a-new-node/94684/2 "2017-07-26T19:56:58Z")
> [@Gkmgaurav](#): > > I had a 3 node cluster on AWS, with ver 1.5 You should _really_ upgrade, 1.5 is no longer supported on any level. > [@Gkmgaurav](#): > > But it is not getting discovered and connecting to my master. What settings have you set? What do the logs show? ---
**Author:** ![Gkmgaurav](https://avatars.discourse-cdn.com/v4/letter/g/91b2a8/32.png) [@Gkmgaurav](https://discuss.elastic.co/u/Gkmgaurav)\ **Post date:** [July 27, 2017, 8:24am UTC](https://discuss.elastic.co/t/unable-to-add-a-new-node/94684/3 "2017-07-27T08:24:58Z")
Below is the cluster status { "cluster\_name" : "air-trends-elk-price-change-cluster", "status" : "yellow", "timed\_out" : false, "number\_of\_nodes" : 1, "number\_of\_data\_nodes" : 1, "active\_primary\_shards" : 671, "active\_shards" : 671, "relocating\_shards" : 0, "initializing\_shards" : 0, "unassigned\_shards" : 671, "number\_of\_pending\_tasks" : 0 } Replication factor is now 1. I will share the logs of Master as well as new node I am trying to add. ![](https://us1.discourse-cdn.com/elastic/original/3X/c/5/c577c5cdf18b933416797f88912901329c1cf745.JPG) ![](https://us1.discourse-cdn.com/elastic/original/3X/2/6/26233e2a5fd5082f38be57b0768c2c0f6f9b10bd.JPG) ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [July 27, 2017, 8:58am UTC](https://discuss.elastic.co/t/unable-to-add-a-new-node/94684/4 "2017-07-27T08:58:25Z")
Don't post images like this. Not readable/searchable. Please update your post. ---
**Author:** ![Gkmgaurav](https://avatars.discourse-cdn.com/v4/letter/g/91b2a8/32.png) [@Gkmgaurav](https://discuss.elastic.co/u/Gkmgaurav)\ **Post date:** [July 27, 2017, 10:18am UTC](https://discuss.elastic.co/t/unable-to-add-a-new-node/94684/5 "2017-07-27T10:18:49Z")
Hi David, the error on new cluster noded, which isnot getting added is like this: ```auto (AbstractChannel.java:206) at org.elasticsearch.transport.netty.NettyTransport.exceptionCaught(NettyTransport.java:611) at org.elasticsearch.transport.netty.MessageChannelHandler.exceptionCaught(MessageChannelHandler.java:239) at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.java:377) at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:74) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) at org.elasticsearch.common.netty.channel.Channels.fireExceptionCaught(Channels.java:525) at org.elasticsearch.common.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:48) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.notifyHandlerException(DefaultChannelPipeline.java:658) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:566) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:74) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268) at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255) at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89) at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:748) [2017-07-24 12:20:41,809][INFO][discovery.ec2] [Moon-Boy] failed to send join request to master [[Master Man][ril_ipI4SYqtlg4Bh3bpnw][ip-10-0-21-253][inet[/10.0.21.253:9300]]{master=true}], reason [RemoteTransportException[[Master Man][inet[/10.0.21.253:9300]][internal:discovery/zen/join]]; nested: ConnectTransportException[[Moon-Boy][inet[/10.0.21.168:9300]] connect_timeout[30s]]; ] [2017-07-24 12:21:32,089][INFO][discovery.ec2] [Moon-Boy] failed to send join request to master [[Master Man][ril_ipI4SYqtlg4Bh3bpnw][ip-10-0-21-253][inet[/10.0.21.253:9300]]{master=true}], reason [RemoteTransportException[[Master Man][inet[/10.0.21.253:9300]][internal:discovery/zen/join]]; nested: ConnectTransportException[[Moon-Boy][inet[/10.0.21.168:9300]] connect_timeout[30s]]; ] [2017-07-24 12:22:22,664][INFO][discovery.ec2] [Moon-Boy] failed to send join request to master [[Master Man][ril_ipI4SYqtlg4Bh3bpnw][ip-10-0-21-253][inet[/10.0.21.253:9300]]{master=true}] ``` ---
**Author:** ![Gkmgaurav](https://avatars.discourse-cdn.com/v4/letter/g/91b2a8/32.png) [@Gkmgaurav](https://discuss.elastic.co/u/Gkmgaurav)\ **Post date:** [July 27, 2017, 10:20am UTC](https://discuss.elastic.co/t/unable-to-add-a-new-node/94684/6 "2017-07-27T10:20:54Z")
the error on master node, looks like below: ```auto (ThreadRenamingRunnable.java:108) at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) [2017-07-24 12:20:08,111][WARN][transport.netty] [Master Man] exception caught on transport layer [[id: 0x15e91eda, /10.0.21.253:33760 :> /10.0.21.253:9300]], closing connection java.io.StreamCorruptedException: invalid internal transport message format, got (ff,f4,ff,fd) at org.elasticsearch.transport.netty.SizeHeaderFrameDecoder.decode(SizeHeaderFrameDecoder.java:47) at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:425) at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.cleanup(FrameDecoder.java:482) at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.channelDisconnected(FrameDecoder.java:365) at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:102) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:74) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) at org.elasticsearch.common.netty.channel.Channels.fireChannelDisconnected(Channels.java:396) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.close(AbstractNioWorker.java:360) at org.elasticsearch.common.netty.channel.socket.nio.NioServerSocketPipelineSink.handleAcceptedSocket(NioServerSocketPipelineSink.java:81) at org.elasticsearch.common.netty.channel.socket.nio.NioServerSocketPipelineSink.eventSunk(NioServerSocketPipelineSink.java:36) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:574) at org.elasticsearch.common.netty.channel.Channels.close(Channels.java:812) at org.elasticsearch.common.netty.channel.AbstractChannel.close(AbstractChannel.java:206) at org.elasticsearch.transport.netty.NettyTransport.exceptionCaught(NettyTransport.java:611) at org.elasticsearch.transport.netty.MessageChannelHandler.exceptionCaught(MessageChannelHandler.java:239) at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.java:377) at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:74) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) at org.elasticsearch.common.netty.channel.Channels.fireExceptionCaught(Channels.java:525) at org.elasticsearch.common.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:48) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.notifyHandlerException(DefaultChannelPipeline.java:658) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:566) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:74) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268) at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255) at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89) at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) ``` ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [July 27, 2017, 11:48pm UTC](https://discuss.elastic.co/t/unable-to-add-a-new-node/94684/7 "2017-07-27T23:48:26Z")
That almost looks like you are joining a node with an incompatible version, are you sure everything is 1.5? ---
**Author:** ![Gkmgaurav](https://avatars.discourse-cdn.com/v4/letter/g/91b2a8/32.png) [@Gkmgaurav](https://discuss.elastic.co/u/Gkmgaurav)\ **Post date:** [August 5, 2017, 10:45am UTC](https://discuss.elastic.co/t/unable-to-add-a-new-node/94684/8 "2017-08-05T10:45:56Z")
Thanks everyone. Issue got resolved. We restarted the master data server and it was able to add the new nodes easily. Thanks to evryone . ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 2, 2017, 10:46am UTC](https://discuss.elastic.co/t/unable-to-add-a-new-node/94684/9 "2017-09-02T10:46:45Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.