# Master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster

**URL:** <https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304>\
**Category:** Elasticsearch\
**Created:** [April 11, 2019, 1:34am UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304 "2019-04-11T01:34:47Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![kyle\_che](https://avatars.discourse-cdn.com/v4/letter/k/ce73a5/32.png) [@kyle\_che](https://discuss.elastic.co/u/kyle_che)\
**Post date:** [April 11, 2019, 1:34am UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/1 "2019-04-11T01:34:47Z")

</div>

I'm trying to upgrade from 6.7.1 to 7.0.0 but getting the below error. I changed discovery.zen.ping.unicast.hosts to discovery.seed\_hosts and added also cluster.initial\_master\_nodes pointing to the same master nodes. What am I missing for this to come up?

curl -XGET "localhost:9200/\_cat/health?v&pretty"  
{  
"error" : {  
"root\_cause" : [  
{  
"type" : "master\_not\_discovered\_exception",  
"reason" : null  
}  
],  
"type" : "master\_not\_discovered\_exception",  
"reason" : null  
},  
"status" : 503  
}

[WARN][o.e.c.c.ClusterFormationFailureHelper] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [April 11, 2019, 7:47am UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/2 "2019-04-11T07:47:45Z")

</div>

Can you share the `elasticsearch.yml` file from one of your master nodes?

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [April 11, 2019, 8:40am UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/3 "2019-04-11T08:40:14Z")

</div>

Also the full warning message from `o.e.c.c.ClusterFormationFailureHelper` (everything, from the timestamp at the start onwards) would be useful to see. You've truncated all the important bits out of your OP!

---

<div class="post-metadata">

**Author:** ![safderali5](https://avatars.discourse-cdn.com/v4/letter/s/8edcca/32.png) [@safderali5](https://discuss.elastic.co/u/safderali5)\
**Post date:** [April 11, 2019, 10:41am UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/4 "2019-04-11T10:41:45Z")

</div>

I have the same issue when migrating from 6.7.0 to 7.0.0

---

<div class="post-metadata">

**Author:** ![skaldenhoven](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/skaldenhoven/32/47233_2.png) [@skaldenhoven](https://discuss.elastic.co/u/skaldenhoven)\
**Post date:** [April 11, 2019, 10:47am UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/5 "2019-04-11T10:47:23Z")

</div>

I had the same.  
Put in the following into your elasticsearch.yml of your master node.  
`cluster.initial_master_nodes:`  
`- node_name_or_ip`

Start the first node and then the rest of the nodes.

---

<div class="post-metadata">

**Author:** ![safderali5](https://avatars.discourse-cdn.com/v4/letter/s/8edcca/32.png) [@safderali5](https://discuss.elastic.co/u/safderali5)\
**Post date:** [April 11, 2019, 10:49am UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/6 "2019-04-11T10:49:55Z")

</div>

I am running on kubernetes, node hostname or IP are not known in advance

---

<div class="post-metadata">

**Author:** ![skaldenhoven](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/skaldenhoven/32/47233_2.png) [@skaldenhoven](https://discuss.elastic.co/u/skaldenhoven)\
**Post date:** [April 11, 2019, 10:52am UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/7 "2019-04-11T10:52:40Z")

</div>

Sorry then I wouldn't know how to solve this. This was on a regular host, no experience (yet) with Kubernetes.

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [April 11, 2019, 10:57am UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/8 "2019-04-11T10:57:16Z")

</div>

> [@safderali5](#):
>
> I am running on kubernetes, node hostname or IP are not known in advance

In this situation you should use the node names, i.e. the values of `node.name`, not the hostnames or IP addresses.

---

<div class="post-metadata">

**Author:** ![safderali5](https://avatars.discourse-cdn.com/v4/letter/s/8edcca/32.png) [@safderali5](https://discuss.elastic.co/u/safderali5)\
**Post date:** [April 11, 2019, 11:04am UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/9 "2019-04-11T11:04:32Z")

</div>

In case of kubernetes, the node.name is same as hostname. as it is name of the pod auto generated by kubernetes. Cannot be known in advance.

---

<div class="post-metadata">

**Author:** ![safderali5](https://avatars.discourse-cdn.com/v4/letter/s/8edcca/32.png) [@safderali5](https://discuss.elastic.co/u/safderali5)\
**Post date:** [April 11, 2019, 11:19am UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/10 "2019-04-11T11:19:37Z")

</div>

[2019-04-11T11:16:43,126][WARN][o.e.d.SeedHostsResolver] [elasticsearch-master-1-vl2l6] failed to resolve host [elasticsearch-master-site1.svc.cluster.local]

java.net.UnknownHostException: elasticsearch-master-site1-rta3.svc.cluster.local

at java.net.InetAddress.getAllByName0(InetAddress.java:1281) ~[?:1.8.0\_202]

at java.net.InetAddress.getAllByName(InetAddress.java:1193) ~[?:1.8.0\_202]

at java.net.InetAddress.getAllByName(InetAddress.java:1127) ~[?:1.8.0\_202]

at org.elasticsearch.transport.TcpTransport.parse(TcpTransport.java:536) ~[elasticsearch-7.0.0.jar:7.0.0]

at org.elasticsearch.transport.TcpTransport.addressesFromString(TcpTransport.java:489) ~[elasticsearch-7.0.0.jar:7.0.0]

at org.elasticsearch.transport.TransportService.addressesFromString(TransportService.java:738) ~[elasticsearch-7.0.0.jar:7.0.0]

at org.elasticsearch.discovery.SeedHostsResolver.lambda$resolveHostsLists$0(SeedHostsResolver.java:143) ~[elasticsearch-7.0.0.jar:7.0.0]

at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0\_202]

at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:681) ~[elasticsearch-7.0.0.jar:7.0.0]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0\_202]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0\_202]

at java.lang.Thread.run(Thread.java:748) [?:1.8.0\_202]

---

<div class="post-metadata">

**Author:** ![safderali5](https://avatars.discourse-cdn.com/v4/letter/s/8edcca/32.png) [@safderali5](https://discuss.elastic.co/u/safderali5)\
**Post date:** [April 11, 2019, 11:21am UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/11 "2019-04-11T11:21:20Z")

</div>

While everything worked with version 6.7.0 with same setup

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [April 11, 2019, 11:24am UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/12 "2019-04-11T11:24:59Z")

</div>

> [@safderali5](#):
>
> In case of kubernetes, the node.name is same as hostname. as it is name of the pod auto generated by kubernetes. Cannot be known in advance.

That's the default for `node.name` but you should be overriding it. See for instance [how the Helm chart does it](https://github.com/elastic/helm-charts/blob/720268f405865cf2974305878a021f4bed27c526/elasticsearch/templates/statefulset.yaml#L154).

---

<div class="post-metadata">

**Author:** ![safderali5](https://avatars.discourse-cdn.com/v4/letter/s/8edcca/32.png) [@safderali5](https://discuss.elastic.co/u/safderali5)\
**Post date:** [April 11, 2019, 11:26am UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/13 "2019-04-11T11:26:30Z")

</div>

I am doing it already to set es node.name to metadata.name of pod

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [April 11, 2019, 11:28am UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/14 "2019-04-11T11:28:45Z")

</div>

Ok, so can you do [what the Helm chart does](https://github.com/elastic/helm-charts/blob/720268f405865cf2974305878a021f4bed27c526/elasticsearch/templates/statefulset.yaml#L160-L161) to set `cluster.initial_master_nodes` too?

---

<div class="post-metadata">

**Author:** ![safderali5](https://avatars.discourse-cdn.com/v4/letter/s/8edcca/32.png) [@safderali5](https://discuss.elastic.co/u/safderali5)\
**Post date:** [April 11, 2019, 11:36am UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/15 "2019-04-11T11:36:45Z")

</div>

I cannot do it, as helm chart is using template helpers to extract that value, In my case i have simple statefulset defined. To do what helm chart is doing, i have to use helm, that means complete revamp of deployment.

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [April 11, 2019, 11:47am UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/16 "2019-04-11T11:47:58Z")

</div>

If you are using a bare `StatefulSet` then AIUI the pod names are predictable:

[https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#stable-network-id](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#stable-network-id)

Is this not the case in your deployment?

---

<div class="post-metadata">

**Author:** ![safderali5](https://avatars.discourse-cdn.com/v4/letter/s/8edcca/32.png) [@safderali5](https://discuss.elastic.co/u/safderali5)\
**Post date:** [April 11, 2019, 12:23pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/17 "2019-04-11T12:23:11Z")

</div>

I cannot rely on statefulset predictable behavior, I will rollback to ES 6.7.0 as this seems to be a major bug in ES 7.0.0

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [April 11, 2019, 12:26pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/18 "2019-04-11T12:26:49Z")

</div>

> [@safderali5](#):
>
> I cannot rely on statefulset predictable behavior

Can you clarify what you mean by this? Do you know of situations where it doesn't behave as documented?

---

<div class="post-metadata">

**Author:** ![kyle\_che](https://avatars.discourse-cdn.com/v4/letter/k/ce73a5/32.png) [@kyle\_che](https://discuss.elastic.co/u/kyle_che)\
**Post date:** [April 11, 2019, 12:28pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/19 "2019-04-11T12:28:15Z")

</div>

Here is my elasticsearch.yml file

cluster.name: dblogging\_es\_dev  
node.name: ${HOSTNAME}  
node.master: true  
node.data: false  
node.ingest: false  
node.max\_local\_storage\_nodes: 1  
path.data: /data/elasticsearch  
path.repo: /dbbackup/d-gp2-es46-1  
path.logs: /var/log/elasticsearch  
network.host: _site_  
network.bind\_host: 0.0.0.0  
transport.tcp.port: 9300  
http.port: 9200  
discovery.zen.minimum\_master\_nodes: 2  
discovery.zen.ping\_timeout: 5s  
discovery.seed\_hosts: ["d-gp2-es46-1", "d-gp2-es46-2", "d-gp2-es46-3"]  
cluster.initial\_master\_nodes: ["d-gp2-es46-1", "d-gp2-es46-2", "d-gp2-es46-3"]  
bootstrap.memory\_lock: true

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [April 11, 2019, 12:30pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/20 "2019-04-11T12:30:23Z")

</div>

Thanks @kyle_che, can you also share the whole `o.e.c.c.ClusterFormationFailureHelper` message?

[Next page](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304.md?page=2)
