# Error when adding node into cluster - Failed to send join request to master

**URL:** <https://discuss.elastic.co/t/error-when-adding-node-into-cluster-failed-to-send-join-request-to-master/55414>\
**Category:** Elasticsearch\
**Created:** [July 13, 2016, 1:57pm UTC](https://discuss.elastic.co/t/error-when-adding-node-into-cluster-failed-to-send-join-request-to-master/55414 "2016-07-13T13:57:14Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![sattu531](https://avatars.discourse-cdn.com/v4/letter/s/8edcca/32.png) [@sattu531](https://discuss.elastic.co/u/sattu531)\
**Post date:** [July 13, 2016, 1:57pm UTC](https://discuss.elastic.co/t/error-when-adding-node-into-cluster-failed-to-send-join-request-to-master/55414/1 "2016-07-13T13:57:14Z")

</div>

Hi,  
I am creating multiple nodes under one cluster. before to set up ES on production Server, i m doing it on my local machine-  
Got tar file of ES and unzipped it and put same in two directorories(D1,D2)  
and also install ES as service on same machine  
Edit config file in all location(D1,D2 and /etc/elasticsearch/elasticsearch.yml) with -  
in D1 - set cluster name as test-cluster and node name as node-1  
in D2 - set cluster name as test-cluster and node name as node-2  
in /etc/elasticsearch/elasticsearch.yml - set cluster name as test-cluster and node name as node-3  
started ES instance in D2 and it is successfully started as **master node**  
now started ES instance in D1 and it is successfully started and joined the running cluster  
now started Es instance as a service and it is successfully started and but not able join cluster and getting following error -  
error in logs on node-3-  
[2016-07-13 19:03:23,093][WARN][transport.netty] [node-3] exception caught on transport layer [[id: 0xc73d8b09, /127.0.0.1:39331 =\> /127.0.0.1:9302]], closing connection  
java.lang.IllegalStateException: Message not fully read (request) for requestId [11508], action [internal:discovery/zen/join/validate], readerIndex [7874] vs expected [9381]; resetting  
at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:121)  
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)  
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)  
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)  
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)  
at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)  
at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)  
at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)  
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)  
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)  
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)  
at org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:75)  
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)  
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)  
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)  
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)  
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)  
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)  
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)  
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)  
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)  
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)  
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)  
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  
at java.lang.Thread.run(Thread.java:745)  
[2016-07-13 19:03:23,096][INFO][discovery.zen] [node-3] failed to send join request to master [{node-2}{nZ1u0minT06lDpOIVC0rhQ}{127.0.0.1}{127.0.0.1:9301}], reason [RemoteTransportException[[node-2][127.0.0.1:9301][internal:discovery/zen/join]]; nested: IllegalStateException[failure when sending a validation request to node]; nested: RemoteTransportException[[node-3][127.0.0.1:9302][internal:discovery/zen/join/validate]]; nested: IllegalArgumentException[No custom metadata prototype registered for type [licenses], node like missing plugins]; ]

and in node-1 logs error is -  
[2016-07-13 19:24:52,579][WARN][discovery.zen] [node-2] failed to validate incoming join request from node [{node-3}{6LgUEyWZQmeYuFoCSZ5EUA}{127.0.0.1}{127.0.0.1:9302}]

Any kind help will be much appreciated.

Thanks,  
Satty

---

<div class="post-metadata">

**Author:** ![sattu531](https://avatars.discourse-cdn.com/v4/letter/s/8edcca/32.png) [@sattu531](https://discuss.elastic.co/u/sattu531)\
**Post date:** [July 13, 2016, 1:58pm UTC](https://discuss.elastic.co/t/error-when-adding-node-into-cluster-failed-to-send-join-request-to-master/55414/2 "2016-07-13T13:58:11Z")

</div>

I am using ES 2.3.2

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [July 13, 2016, 2:09pm UTC](https://discuss.elastic.co/t/error-when-adding-node-into-cluster-failed-to-send-join-request-to-master/55414/3 "2016-07-13T14:09:03Z")

</div>

Apparently you did not install license plugin on all nodes? `No custom metadata prototype registered for type [licenses], node like missing plugins`

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [July 13, 2016, 2:09pm UTC](https://discuss.elastic.co/t/error-when-adding-node-into-cluster-failed-to-send-join-request-to-master/55414/4 "2016-07-13T14:09:39Z")

</div>

Oh and BTW why do you install 2 nodes on the same machine?

---

<div class="post-metadata">

**Author:** ![sattu531](https://avatars.discourse-cdn.com/v4/letter/s/8edcca/32.png) [@sattu531](https://discuss.elastic.co/u/sattu531)\
**Post date:** [July 13, 2016, 2:16pm UTC](https://discuss.elastic.co/t/error-when-adding-node-into-cluster-failed-to-send-join-request-to-master/55414/5 "2016-07-13T14:16:32Z")

</div>

Hi Dadoonet,  
I am doing it on my local machine and to test this scenario .. i need to test it on my local machine.

So i m running 3 ES instances on my machine and all are running fine but one instance which is started as a service which config file located in "/etc/elasticsearch" is not joining cluster.

I went through ES doc to add nodes(running on different machines) into cluster- [https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html#common-network-settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html#common-network-settings)

It is saying no need to change network config settings if all nodes are running on same machine.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [July 13, 2016, 2:55pm UTC](https://discuss.elastic.co/t/error-when-adding-node-into-cluster-failed-to-send-join-request-to-master/55414/6 "2016-07-13T14:55:29Z")

</div>

So don't run instances as services. Just run them from the elasticsearch directory.

Basically launch in 2 windows:

```auto
bin/elasticsearch

```

And that's all.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:36pm UTC](https://discuss.elastic.co/t/error-when-adding-node-into-cluster-failed-to-send-join-request-to-master/55414/7 "2017-07-05T22:36:02Z")

</div>


