# Data nodes are connected to the master but not to each other during upgrade

**URL:** <https://discuss.elastic.co/t/data-nodes-are-connected-to-the-master-but-not-to-each-other-during-upgrade/278472>\
**Category:** Elasticsearch\
**Created:** [July 12, 2021, 10:35pm UTC](https://discuss.elastic.co/t/data-nodes-are-connected-to-the-master-but-not-to-each-other-during-upgrade/278472 "2021-07-12T22:35:13Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![163satish](https://avatars.discourse-cdn.com/v4/letter/1/7ab992/32.png) [@163satish](https://discuss.elastic.co/u/163satish)\
**Post date:** [July 12, 2021, 10:35pm UTC](https://discuss.elastic.co/t/data-nodes-are-connected-to-the-master-but-not-to-each-other-during-upgrade/278472/1 "2021-07-12T22:35:13Z")

</div>

I am doing ES upgrade from 6.8.2 to 7.9.3 and seeing issues with 2 data nodes. Shard movements are going from "index" to "init" stage and logs show that the data nodes are not connected to each other.  
On the new node I see NodeNotConnectedException:

```auto
org.elasticsearch.transport.NodeNotConnectedException: [xx.xx.xx.xx:9300] Node not connected
        at org.elasticsearch.transport.ClusterConnectionManager.getConnection(ClusterConnectionManager.java:189) ~[elasticsearch-7.9.3.jar:7.9.3]
        at org.elasticsearch.transport.TransportService.getConnection(TransportService.java:673) ~[elasticsearch-7.9.3.jar:7.9.3]
        at org.elasticsearch.action.search.SearchTransportService.getConnection(SearchTransportService.java:373) ~[elasticsearch-7.9.3.jar:7.9.3]
        at org.elasticsearch.action.search.TransportSearchAction.lambda$buildConnectionLookup$6(TransportSearchAction.java:537) ~[elasticsearch-7.9.3.jar:7.9.3]
        at org.elasticsearch.action.search.AbstractSearchAsyncAction.getConnection(AbstractSearchAsyncAction.java:587) ~[elasticsearch-7.9.3.jar:7.9.3]
        at org.elasticsearch.action.search.SearchQueryThenFetchAsyncAction.executePhaseOnShard(SearchQueryThenFetchAsyncAction.java:79) ~[elasticsearch-7.9.3.jar:7.9.3]

```

While on old data nodes, I see this:

```auto
[2021-07-12T22:31:53,209][WARN][o.e.t.OutboundHandler] [xxxxxx-xxxxxx] send message failed [channel: Netty4TcpChannel{localAddress=xx.xx.xx.xx/xx.xx.xx.xx:9300, remoteAddress=/xx.xx.xx.xx:34692}]
java.nio.channels.ClosedChannelException: null
	at io.netty.channel.AbstractChannel$AbstractUnsafe.write(...)(Unknown Source) ~[?:?]

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 9, 2021, 10:35pm UTC](https://discuss.elastic.co/t/data-nodes-are-connected-to-the-master-but-not-to-each-other-during-upgrade/278472/2 "2021-08-09T22:35:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.