# The node does not automatically return to the Elasticsearch cluster

**URL:** https://discuss.elastic.co/t/the-node-does-not-automatically-return-to-the-elasticsearch-cluster/251848
**Category:** Elasticsearch
**Created:** [October 13, 2020, 6:29am UTC](https://discuss.elastic.co/t/the-node-does-not-automatically-return-to-the-elasticsearch-cluster/251848 "2020-10-13T06:29:38Z")
**Posts on this page:** 20
**Page:** 1

<div class="post-metadata">

### Author: ![djuwa4](https://avatars.discourse-cdn.com/v4/letter/d/a8b319/32.png) [@djuwa4](https://discuss.elastic.co/u/djuwa4)
#### Post date: [October 13, 2020, 6:29am UTC](https://discuss.elastic.co/t/the-node-does-not-automatically-return-to-the-elasticsearch-cluster/251848/1 "2020-10-13T06:29:39Z")

</div>

Good afternoon! I have a cluster of five nodes: three node.master and all five node.data. There is a problem when testing it. If you turn off one node and turn it on again, its elasticsearch instance does not see the cluster.  
Only the simultaneous restart of the elasticsearch service on all nodes helps.

Here is the node config:

```auto
cluster.name: lynx
node.name: phd-1
node.master: true
node.data: true
network.bind_host: 0.0.0.0
network.host: 10.10.10.15
http.port: 9200
discovery.zen.ping.unicast.hosts: [“10.10.10.11”, “10.10.10.12”, “10.10.10.13”, “10.10.10.14”, “10.10.10.15”]
discovery.zen.minimum_master_nodes: 2

```

Please tell me what I missed and how can I fix this problem?

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [October 13, 2020, 6:46am UTC](https://discuss.elastic.co/t/the-node-does-not-automatically-return-to-the-elasticsearch-cluster/251848/2 "2020-10-13T06:46:01Z")

</div>

Can you please provide the output of the `_cat/nodes` API?

---

<div class="post-metadata">

### Author: ![djuwa4](https://avatars.discourse-cdn.com/v4/letter/d/a8b319/32.png) [@djuwa4](https://discuss.elastic.co/u/djuwa4)
#### Post date: [October 13, 2020, 6:46am UTC](https://discuss.elastic.co/t/the-node-does-not-automatically-return-to-the-elasticsearch-cluster/251848/3 "2020-10-13T06:46:12Z")

</div>

If you enter the command on this node: curl -XGET 'http: // localhost: 9200 / \_cluster / health'  
I see this error:

```auto
{"error":{"root_cause":[{"type":"master_not_discovered_exception","reason":null}],"type":"master_not_discovered_exception","reason":null},"status":503}root@phd- 

```

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [October 13, 2020, 6:47am UTC](https://discuss.elastic.co/t/the-node-does-not-automatically-return-to-the-elasticsearch-cluster/251848/4 "2020-10-13T06:47:49Z")

</div>

What about the other nodes? Which version of Elasticsearch are you using?

---

<div class="post-metadata">

### Author: ![djuwa4](https://avatars.discourse-cdn.com/v4/letter/d/a8b319/32.png) [@djuwa4](https://discuss.elastic.co/u/djuwa4)
#### Post date: [October 13, 2020, 7:07am UTC](https://discuss.elastic.co/t/the-node-does-not-automatically-return-to-the-elasticsearch-cluster/251848/5 "2020-10-13T07:07:17Z")

</div>

I hope I am doing it right:

```auto
curl http://localhost:9200/_cat/nodes

```

The error is the same as before:

```auto
{"error":{"root_cause":[{"type":"master_not_discovered_exception","reason":null}],"type":"maste

```

When I run the command: **curl -XGET 'http: [//localhost](https://localhost): 9200/\_cluster/helten** on other nodes:

```auto
{"cluster_name":"lynx_new","status":"green","timed_out":false,"number_of_nodes":4,"number_of_data_nodes":4,"active_primary_shards":0,"active_shards":0,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":100.0}root@poi-u1:/opt

```

Product version: 7.7.2

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [October 13, 2020, 7:25am UTC](https://discuss.elastic.co/t/the-node-does-not-automatically-return-to-the-elasticsearch-cluster/251848/6 "2020-10-13T07:25:44Z")

</div>

> [@djuwa4](#):
>
> ```auto
> discovery.zen.ping.unicast.hosts: [“10.10.10.11”, “10.10.10.12”, “10.10.10.13”, “10.10.10.14”, “10.10.10.15”]
> discovery.zen.minimum_master_nodes: 2
> 
> ```

If you are using Elasticsearch 7.7 these settings are not the correct ones as cluster configuration changed in Elasticsearch 7.x. I suspect you may need to set up the cluster again, and when you do so make sure you are looking at [the docs belonging to the correct version](https://www.elastic.co/guide/en/elasticsearch/reference/7.7/modules-discovery-settings.html).

---

<div class="post-metadata">

### Author: ![djuwa4](https://avatars.discourse-cdn.com/v4/letter/d/a8b319/32.png) [@djuwa4](https://discuss.elastic.co/u/djuwa4)
#### Post date: [October 13, 2020, 7:32am UTC](https://discuss.elastic.co/t/the-node-does-not-automatically-return-to-the-elasticsearch-cluster/251848/7 "2020-10-13T07:32:56Z")

</div>

Thanks!

---

<div class="post-metadata">

### Author: ![djuwa4](https://avatars.discourse-cdn.com/v4/letter/d/a8b319/32.png) [@djuwa4](https://discuss.elastic.co/u/djuwa4)
#### Post date: [October 13, 2020, 8:43am UTC](https://discuss.elastic.co/t/the-node-does-not-automatically-return-to-the-elasticsearch-cluster/251848/8 "2020-10-13T08:43:32Z")

</div>

Apparently, I gave you the wrong version of the product, version:

```auto
"version": {
     "number": "6.8.6"

```

Full output command curl -XGET 'localhost:9200':

```auto
{
  "name" : "phd-1",
  "cluster_name" : "lynx_new",
  "cluster_uuid" : "_na_",
  "version" : {
    "number" : "6.8.6",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "3d9f765",
    "build_date" : "2019-12-13T17:11:52.013738Z",
    "build_snapshot" : false,
    "lucene_version" : "7.7.2",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

```

It turns out that the problem persists. I even disabled the firewall completely during testing.  
I apologize for initially making a mistake with the version!

**curl [http://localhost:9200/\_cat/nodes](http://localhost:9200/_cat/nodes)**:

```auto
10.10.10.12 33 67 3 0.28 0.39 0.36 di - psis-1
10.10.10.13 32 52 2 0.31 0.37 0.36 mdi - poi-1
10.10.10.14 31 43 3 0.29 0.56 0.47 di - pdi-1
10.10.10.11 33 73 3 0.32 1.85 1.55 mdi * pfur-1

```

---

<div class="post-metadata">

### Author: ![djuwa4](https://avatars.discourse-cdn.com/v4/letter/d/a8b319/32.png) [@djuwa4](https://discuss.elastic.co/u/djuwa4)
#### Post date: [October 13, 2020, 11:18am UTC](https://discuss.elastic.co/t/the-node-does-not-automatically-return-to-the-elasticsearch-cluster/251848/10 "2020-10-13T11:18:26Z")

</div>

No options on how to fix this problem?

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [October 13, 2020, 11:23am UTC](https://discuss.elastic.co/t/the-node-does-not-automatically-return-to-the-elasticsearch-cluster/251848/11 "2020-10-13T11:23:30Z")

</div>

I see two different cluster names. Is that due to you changing it at some point or does it differ between the nodes?

---

<div class="post-metadata">

### Author: ![djuwa4](https://avatars.discourse-cdn.com/v4/letter/d/a8b319/32.png) [@djuwa4](https://discuss.elastic.co/u/djuwa4)
#### Post date: [October 13, 2020, 11:39am UTC](https://discuss.elastic.co/t/the-node-does-not-automatically-return-to-the-elasticsearch-cluster/251848/12 "2020-10-13T11:39:40Z")

</div>

Yes, I changed the cluster name for all nodes. I double-checked, now all nodes have a name: cluster.name: lynx\_new  
When I first assembled a cluster he had the name lynx. I rebuilt it with a new name, without changing other parameters.  
From the problem node, I checked access to one of the master nodes located in the cluster:

```auto
telnet 10.10.10.13 9300
Trying 10.10.10.13...
Connected to 10.10.10.13.
Escape character is '^]'.

```

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [October 13, 2020, 11:48am UTC](https://discuss.elastic.co/t/the-node-does-not-automatically-return-to-the-elasticsearch-cluster/251848/13 "2020-10-13T11:48:39Z")

</div>

Can it reach all the other nodes the same way? Can the other nodes reach this node?

---

<div class="post-metadata">

### Author: ![djuwa4](https://avatars.discourse-cdn.com/v4/letter/d/a8b319/32.png) [@djuwa4](https://discuss.elastic.co/u/djuwa4)
#### Post date: [October 13, 2020, 11:58am UTC](https://discuss.elastic.co/t/the-node-does-not-automatically-return-to-the-elasticsearch-cluster/251848/14 "2020-10-13T11:58:57Z")

</div>

Yes, this host sees other nodes, these nodes see this host.  
I looked at the cluster log file /var/log/elasticsearch/lynx\_new.log  
I am confused by these errors:

```auto
[phd-1] failed to resolve host [“10.10.10.11”]

UnicastZenPing ] [phd-1] failed to resolve host [“10.10.10.12”]

[o.e.d.z.UnicastZenPing] [phd-1] failed to resolve host [“10.10.10.12”]
java.net.UnknownHostException: “10.10.10.12”

```

Perhaps this entry is incorrect:

```auto
discovery.zen.ping.unicast.hosts: [“10.10.10.11”, “10.10.10.12”, “10.10.10.13”, “10.10.10.14”, “10.10.10.15”]

```

---

<div class="post-metadata">

### Author: ![djuwa4](https://avatars.discourse-cdn.com/v4/letter/d/a8b319/32.png) [@djuwa4](https://discuss.elastic.co/u/djuwa4)
#### Post date: [October 13, 2020, 12:31pm UTC](https://discuss.elastic.co/t/the-node-does-not-automatically-return-to-the-elasticsearch-cluster/251848/15 "2020-10-13T12:31:24Z")

</div>

Yes, it seems to be so, the entry:

```auto
[“10.10.10.11”, “10.10.10.12”, “10.10.10.13”, “10.10.10.14”, “10.10.10.15”] 

```

WRONG  
I did this:

```auto
['10.10.10.11', '10.10.10.12', '10.10.10.13', '10.10.10.14', '10.10.10.15']

```

And it works

---

<div class="post-metadata">

### Author: ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)
#### Post date: [October 13, 2020, 1:09pm UTC](https://discuss.elastic.co/t/the-node-does-not-automatically-return-to-the-elasticsearch-cluster/251848/16 "2020-10-13T13:09:58Z")

</div>

Yes, the mistake was that you were using `“` ([`LEFT DOUBLE QUOTATION MARK`](https://www.fileformat.info/info/unicode/char/201c/index.htm)) and `”` ([`RIGHT DOUBLE QUOTATION MARK`](https://www.fileformat.info/info/unicode/char/201d/index.htm)) rather than `"` ([`QUOTATION MARK`](https://www.fileformat.info/info/unicode/char/22/index.htm)) around the IP addresses.

---

<div class="post-metadata">

### Author: ![djuwa4](https://avatars.discourse-cdn.com/v4/letter/d/a8b319/32.png) [@djuwa4](https://discuss.elastic.co/u/djuwa4)
#### Post date: [October 13, 2020, 1:27pm UTC](https://discuss.elastic.co/t/the-node-does-not-automatically-return-to-the-elasticsearch-cluster/251848/17 "2020-10-13T13:27:24Z")

</div>

I want to know if the following behavior is normal: when I shutdown two out of three master nodes, my cluster fall apart? I believed that the cluster will work with one master node.  
I have disabled two master nodes from three and one node data. All five nodes - data nodes. On the two remaining nodes, I get the error:

```auto
{"error":{"root_cause":[{"type":"master_not_discovered_exception","reason":null}],"type":"master_not_discovered_exception","reason":null},"status":503}root@pdi-

```

After executing the command:

```auto
curl -XGET 'http://localhost:9200/_cluster/health'

```

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [October 13, 2020, 1:33pm UTC](https://discuss.elastic.co/t/the-node-does-not-automatically-return-to-the-elasticsearch-cluster/251848/18 "2020-10-13T13:33:39Z")

</div>

You always need a majority of master eligible nodes present for a healthy cluster, so what you are seeing is expected.

---

<div class="post-metadata">

### Author: ![djuwa4](https://avatars.discourse-cdn.com/v4/letter/d/a8b319/32.png) [@djuwa4](https://discuss.elastic.co/u/djuwa4)
#### Post date: [October 13, 2020, 1:40pm UTC](https://discuss.elastic.co/t/the-node-does-not-automatically-return-to-the-elasticsearch-cluster/251848/19 "2020-10-13T13:40:20Z")

</div>

Yes, I understood this, because I specified the parameter: discovery.zen.minimum\_master\_nodes: 2  
Based on this, a cluster requires at least two master nodes.

---

<div class="post-metadata">

### Author: ![djuwa4](https://avatars.discourse-cdn.com/v4/letter/d/a8b319/32.png) [@djuwa4](https://discuss.elastic.co/u/djuwa4)
#### Post date: [October 13, 2020, 1:41pm UTC](https://discuss.elastic.co/t/the-node-does-not-automatically-return-to-the-elasticsearch-cluster/251848/20 "2020-10-13T13:41:14Z")

</div>

Thank you very much!

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 10, 2020, 1:41pm UTC](https://discuss.elastic.co/t/the-node-does-not-automatically-return-to-the-elasticsearch-cluster/251848/21 "2020-11-10T13:41:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
