# New Node Not Talking to Cluster

**URL:** <https://discuss.elastic.co/t/new-node-not-talking-to-cluster/42737>\
**Category:** Elasticsearch\
**Created:** [February 25, 2016, 4:13pm UTC](https://discuss.elastic.co/t/new-node-not-talking-to-cluster/42737 "2016-02-25T16:13:55Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Bradly](https://avatars.discourse-cdn.com/v4/letter/b/90ced4/32.png) [@Bradly](https://discuss.elastic.co/u/Bradly)\
**Post date:** [February 25, 2016, 4:13pm UTC](https://discuss.elastic.co/t/new-node-not-talking-to-cluster/42737/1 "2016-02-25T16:13:55Z")

</div>

I added a new node yesterday afternoon and got to a point where it should have been communicating with the cluster but its not.

I receive this error on the new node in the elasticsearch.log file:  
[2016-02-25 09:33:53,802][WARN][transport.netty] [Node-5] Message not fully read (response) for [2698] handler org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$4@2bc2db6d, error [true], resetting  
[2016-02-25 09:33:53,803][WARN][discovery.zen.ping.unicast] [KIBANA-ALPHA] failed to send ping to [[#zen\_unicast\_2#][Node-5][inet[/172.16.0.4:9300]]]  
org.elasticsearch.transport.RemoteTransportException: Failed to deserialize exception response from stream  
Caused by: org.elasticsearch.transport.TransportSerializationException: Failed to deserialize exception response from stream  
at org.elasticsearch.transport.netty.MessageChannelHandler.handlerResponseError(MessageChannelHandler.java:178)

I get this error on the other end (172.16.0.4):  
[2016-02-25 09:34:24,545][WARN][transport.netty] [ELASTICSEARSH-ALPHA] exception caught on transport layer [[id: 0x9626bf34, /172.31.0.3:43864 =\> /172.31.0.65:9300]], closing connection  
java.lang.IllegalStateException: Message not fully read (request) for requestId [2770], action [internal:discovery/zen/unicast\_gte\_1\_4], readerIndex [59] vs expected [199]; resetting  
at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived  
After searching I turned up this command:  
curl 'localhost:9200/\_nodes/jvm?pretty'

On the new node I get:  
{  
"cluster\_name" : "ELASTICSEARCH",  
"nodes" : {  
"lAPD4ZLtT3Kzcj38e9hf-g" : {  
"name" : "NODE-5",  
"transport\_address" : "inet[/172.16.0.5:9300]",  
"host" : "KIBANA-ALPHA",  
"ip" : "127.0.1.1",  
"version" : "1.6.2",  
"build" : "NA",  
"http\_address" : "inet[/172.16.0.5:9200]",  
"attributes" : {  
"data" : "false",  
"master" : "false"  
},  
"jvm" : {  
"pid" : 1801,  
"version" : "1.7.0\_95",  
"vm\_name" : "OpenJDK 64-Bit Server VM",  
"vm\_version" : "24.95-b01",  
"vm\_vendor" : "Oracle Corporation",  
"start\_time\_in\_millis" : 1456413210741,  
"mem" : {  
"heap\_init\_in\_bytes" : 268435456,  
"heap\_max\_in\_bytes" : 1065025536,  
"non\_heap\_init\_in\_bytes" : 24313856,  
"non\_heap\_max\_in\_bytes" : 224395264,  
"direct\_max\_in\_bytes" : 1065025536  
},  
"gc\_collectors" : ["Copy", "ConcurrentMarkSweep"],  
"memory\_pools" : ["Code Cache", "Eden Space", "Survivor Space", "CMS Old Gen", "CMS Perm Gen"]  
}  
}  
}  
}

On the other nodes I show:  
{  
"cluster\_name" : "ELASTICSEARCH",  
"nodes" : {  
"xQXVmG3UTQKJ0lcgEjLR7g" : {  
"name" : "NODE1",  
"transport\_address" : "172.16.0.1:9300",  
"host" : "172.16.0.1",  
"ip" : "172.16.0.1",  
"version" : "2.2.0",  
"build" : "8ff36d1",  
"http\_address" : "172.16.0.1:9200",  
"jvm" : {  
"pid" : 1490,  
"version" : "1.8.0\_72",  
"vm\_name" : "Java HotSpot(TM) 64-Bit Server VM",  
"vm\_version" : "25.72-b15",  
"vm\_vendor" : "Oracle Corporation",  
"start\_time\_in\_millis" : 1456347785056,  
"mem" : {  
"heap\_init\_in\_bytes" : 268435456,  
"heap\_max\_in\_bytes" : 1056309248,  
"non\_heap\_init\_in\_bytes" : 2555904,  
"non\_heap\_max\_in\_bytes" : 0,  
"direct\_max\_in\_bytes" : 1056309248  
},  
"gc\_collectors" : ["ParNew", "ConcurrentMarkSweep"],  
"memory\_pools" : ["Code Cache", "Metaspace", "Compressed Class Space", "Par Eden Space", "Par Survivor Space", "CMS Old Gen"],  
"using\_compressed\_ordinary\_object\_pointers" : "true"  
}  
},

The only difference I see is the lines on transport address read "transport\_address" : "inet[/172.16.0.5:9300]", on the new server and "transport\_address" : "172.16.0.1:9300", on the old servers. Any help would be appreciated.

Brad

---

<div class="post-metadata">

**Author:** ![crickes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/crickes/32/18009_2.png) [@crickes](https://discuss.elastic.co/u/crickes)\
**Post date:** [February 25, 2016, 4:22pm UTC](https://discuss.elastic.co/t/new-node-not-talking-to-cluster/42737/2 "2016-02-25T16:22:36Z")

</div>

Hi,

Your new node is trying to come up on ES 1.7.2 where the rest of your cluster is on 2.2.0? Your new node are also using a much newer version of JAVA (1.8.0\_72 vs 1.7.0\_95). If I was in your shoes, I would get the new node to run the same version of both Java and ES before trying to go any further to rule out any issues between incompatible verions.

Steve

---

<div class="post-metadata">

**Author:** ![Bradly](https://avatars.discourse-cdn.com/v4/letter/b/90ced4/32.png) [@Bradly](https://discuss.elastic.co/u/Bradly)\
**Post date:** [February 25, 2016, 7:57pm UTC](https://discuss.elastic.co/t/new-node-not-talking-to-cluster/42737/3 "2016-02-25T19:57:06Z")

</div>

Awesome. Updated the new one to 2.2.0 and off it went. Didn't even see that when I was reading through it.

Thanks,  
Brad

---

<div class="post-metadata">

**Author:** ![crickes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/crickes/32/18009_2.png) [@crickes](https://discuss.elastic.co/u/crickes)\
**Post date:** [February 25, 2016, 8:06pm UTC](https://discuss.elastic.co/t/new-node-not-talking-to-cluster/42737/4 "2016-02-25T20:06:19Z")

</div>

If you haven't done so yet, I'd recommend upgrading the Java version on the new node to match the rest of your cluster also.

---

<div class="post-metadata">

**Author:** ![Bradly](https://avatars.discourse-cdn.com/v4/letter/b/90ced4/32.png) [@Bradly](https://discuss.elastic.co/u/Bradly)\
**Post date:** [February 25, 2016, 8:32pm UTC](https://discuss.elastic.co/t/new-node-not-talking-to-cluster/42737/5 "2016-02-25T20:32:23Z")

</div>

I will get that done this evening when the firewalls are not shooting quite so much data at the cluster. Do you recommend openjdk or oracle?? I think right now I am running openjdk but if I am going to upgrade it might be a good time to make the switch if there is a good solid argument for a change.

---

<div class="post-metadata">

**Author:** ![crickes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/crickes/32/18009_2.png) [@crickes](https://discuss.elastic.co/u/crickes)\
**Post date:** [February 25, 2016, 8:48pm UTC](https://discuss.elastic.co/t/new-node-not-talking-to-cluster/42737/6 "2016-02-25T20:48:56Z")

</div>

I haven't tried openJDK. My cluster is currently running jre-8u72-linux-x64.rpm and I haven't had any issues. I believe they are up to release jre-8u74-linux-x64.rpm.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:13pm UTC](https://discuss.elastic.co/t/new-node-not-talking-to-cluster/42737/7 "2017-07-05T23:13:34Z")

</div>


