# Elaticsearch always have a node disconnected when initializing shards

**URL:** <https://discuss.elastic.co/t/elaticsearch-always-have-a-node-disconnected-when-initializing-shards/99994>\
**Category:** Elasticsearch\
**Created:** [September 11, 2017, 4:18am UTC](https://discuss.elastic.co/t/elaticsearch-always-have-a-node-disconnected-when-initializing-shards/99994 "2017-09-11T04:18:27Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![tkg\_cangkul](https://avatars.discourse-cdn.com/v4/letter/t/8491ac/32.png) [@tkg\_cangkul](https://discuss.elastic.co/u/tkg_cangkul)\
**Post date:** [September 11, 2017, 4:18am UTC](https://discuss.elastic.co/t/elaticsearch-always-have-a-node-disconnected-when-initializing-shards/99994/1 "2017-09-11T04:18:28Z")

</div>

Hi,

i have 4 nodes for ES in my cluster.  
but there is always a node disconnected when the cluster initializing shard or when i add some configuration after rolling restart.

i have 64Gb RAM on each node.  
is there any solution about this?

Best Regards,

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [September 11, 2017, 7:06am UTC](https://discuss.elastic.co/t/elaticsearch-always-have-a-node-disconnected-when-initializing-shards/99994/2 "2017-09-11T07:06:12Z")

</div>

This is not sufficient information to provide a useful answer.

Have you checked the logfile of the affected node. Also any special configuration should be included in your post, as well as the Elasticsearch version and in the best case steps to reproduce this issue.

Please read how to write [an informative post](https://www.elastic.co/help), so that others can chime in and help.

--Alex

---

<div class="post-metadata">

**Author:** ![tkg\_cangkul](https://avatars.discourse-cdn.com/v4/letter/t/8491ac/32.png) [@tkg\_cangkul](https://discuss.elastic.co/u/tkg_cangkul)\
**Post date:** [September 11, 2017, 7:27am UTC](https://discuss.elastic.co/t/elaticsearch-always-have-a-node-disconnected-when-initializing-shards/99994/3 "2017-09-11T07:27:49Z")

</div>

hi,

thx for your reply.

i'm using elasticsearch 2.3.3 and here is some logs that i've found :

```
[2017-09-11 13:08:14,289][INFO][discovery.zen] [Orka] master_left [{Iron Cross}{6k1R1wJVQZ67D2F0wdd-OA}{10.1.80.226}{10.1.80.226:9300}], reason [transport disconnected]
[2017-09-11 13:08:14,301][WARN][discovery.zen] [Orka] master left (reason = transport disconnected), current nodes: {{Unuscione}{kGsXwmT_R6CJ_tyLfBxM7Q}{10.1.80.220}{10.1.80.220:9300},{Orka}{zWlA3gmbTI6Ug1hgBhecnA}{10.1.80.221}{10.1.80.221:9300},{Doctor Leery}{Y_0ZCdv2TqiDaAvRW9jtgg}{10.1.80.223}{10.1.80.223:9300},}
[2017-09-11 13:08:14,301][INFO][cluster.service] [Orka] removed {{Iron Cross}{6k1R1wJVQZ67D2F0wdd-OA}{10.1.80.226}{10.1.80.226:9300},}, reason: zen-disco-master_failed ({Iron Cross}{6k1R1wJVQZ67D2F0wdd-OA}{10.1.80.226}{10.1.80.226:9300})

```

i have checked on my network but there is no problem about it. the datanode still up when i check it from ambari.  
and this is my configuration :

```
http.port: 9200
transport.tcp.port: 9300
discovery.zen.minimum_master_nodes: 1
discovery.zen.ping.unicast.hosts: ["node1","node2","node3","node4"]
discovery.zen.ping_timeout: 30s
```

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [September 11, 2017, 10:01am UTC](https://discuss.elastic.co/t/elaticsearch-always-have-a-node-disconnected-when-initializing-shards/99994/4 "2017-09-11T10:01:34Z")

</div>

What do you mean with 'when I check it from ambari'? Do you see different 'world' from different hosts? Can you check the logfiles on all nodes, if they detect if the master node is gone?

Also, minimum master nodes should be 3 in this setup, not 1, if all nodes are master node eligible.

---

<div class="post-metadata">

**Author:** ![tkg\_cangkul](https://avatars.discourse-cdn.com/v4/letter/t/8491ac/32.png) [@tkg\_cangkul](https://discuss.elastic.co/u/tkg_cangkul)\
**Post date:** [September 11, 2017, 10:18am UTC](https://discuss.elastic.co/t/elaticsearch-always-have-a-node-disconnected-when-initializing-shards/99994/5 "2017-09-11T10:18:41Z")

</div>

Sorry i mean when i check the Elasticsearch service status on all nodes, the service status is still running. i said from ambari because i install the elasticsearch on ambari.

could you explain me about this please?

> Also, minimum master nodes should be 3 in this setup, not 1, if all nodes are master node eligible.

on my cluster. now i've set 2 master node and 4 datanode

i.e

```
node 1 : master + datanode
node 2 : master + datanode
node 3 : datanode
node 4 : datanode

```

is there any mistake from my architecture or my configuration?  
pls advice,

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [September 11, 2017, 10:36am UTC](https://discuss.elastic.co/t/elaticsearch-always-have-a-node-disconnected-when-initializing-shards/99994/6 "2017-09-11T10:36:51Z")

</div>

You should always aim to have 3 master eligible nodes, with minimum\_master\_nodes set to 2 [in order to avoid split-brain scenarios](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/modules-node.html#split-brain).

---

<div class="post-metadata">

**Author:** ![tkg\_cangkul](https://avatars.discourse-cdn.com/v4/letter/t/8491ac/32.png) [@tkg\_cangkul](https://discuss.elastic.co/u/tkg_cangkul)\
**Post date:** [September 11, 2017, 11:02am UTC](https://discuss.elastic.co/t/elaticsearch-always-have-a-node-disconnected-when-initializing-shards/99994/7 "2017-09-11T11:02:03Z")

</div>

ok i'll try it.  
many thanks before for your suggestion

Best Regards,

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [September 11, 2017, 12:01pm UTC](https://discuss.elastic.co/t/elaticsearch-always-have-a-node-disconnected-when-initializing-shards/99994/8 "2017-09-11T12:01:46Z")

</div>

if you just check for status, then there is no guarantee that those nodes can connect to each other, this is the crucial part.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 9, 2017, 12:02pm UTC](https://discuss.elastic.co/t/elaticsearch-always-have-a-node-disconnected-when-initializing-shards/99994/9 "2017-10-09T12:02:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
