# Master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster

**URL:** <https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304>\
**Category:** Elasticsearch\
**Created:** [April 11, 2019, 1:34am UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304 "2019-04-11T01:34:47Z")\
**Posts on this page:** 20\
**Page:** 3

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [April 11, 2019, 1:46pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/41 "2019-04-11T13:46:48Z")

</div>

Ok, that looks like it succeeded:

```nohighlight
[2019-04-11T13:41:51,674][INFO][o.e.c.s.ClusterApplierService] [d-gp2-kyles-1] master node changed {previous [], current [{d-gp2-kyles-3}{DFvnIfwDRS-g1Z3nGRuogg}{0HE_55jUTOSQDJImo0GcZA}{10.124.193.72}{10.124.193.72:9300}{ml.machine_memory=4143783936, ml.max_open_jobs=20, xpack.installed=true}]}, added {{d-gp2-kyles-3}{DFvnIfwDRS-g1Z3nGRuogg}{0HE_55jUTOSQDJImo0GcZA}{10.124.193.72}{10.124.193.72:9300}{ml.machine_memory=4143783936, ml.max_open_jobs=20, xpack.installed=true},}, term: 1, version: 1, reason: ApplyCommitRequest{term=1, version=1, sourceNode={d-gp2-kyles-3}{DFvnIfwDRS-g1Z3nGRuogg}{0HE_55jUTOSQDJImo0GcZA}{10.124.193.72}{10.124.193.72:9300}{ml.machine_memory=4143783936, ml.max_open_jobs=20, xpack.installed=true}}

```

So now the question is, why doesn't this work for you with `${HOSTNAME}`?

---

<div class="post-metadata">

**Author:** ![kyle\_che](https://avatars.discourse-cdn.com/v4/letter/k/ce73a5/32.png) [@kyle\_che](https://discuss.elastic.co/u/kyle_che)\
**Post date:** [April 11, 2019, 1:46pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/42 "2019-04-11T13:46:49Z")

</div>

actually shows it's working now

---

<div class="post-metadata">

**Author:** ![kyle\_che](https://avatars.discourse-cdn.com/v4/letter/k/ce73a5/32.png) [@kyle\_che](https://discuss.elastic.co/u/kyle_che)\
**Post date:** [April 11, 2019, 1:48pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/43 "2019-04-11T13:48:24Z")

</div>

yeah, that's weird ... and why it didn't work when i was doing my upgrade ... i believe i tried it with both ${HOSTNAME} and the actual name.

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [April 11, 2019, 1:49pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/44 "2019-04-11T13:49:09Z")

</div>

Can you try `echo -n $HOSTNAME | xxd` to see if there's any weird characters in there that aren't being logged faithfully?

---

<div class="post-metadata">

**Author:** ![kyle\_che](https://avatars.discourse-cdn.com/v4/letter/k/ce73a5/32.png) [@kyle\_che](https://discuss.elastic.co/u/kyle_che)\
**Post date:** [April 11, 2019, 1:49pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/45 "2019-04-11T13:49:49Z")

</div>

00000000: 642d 6770 322d 6b79 6c65 732d 31 d-gp2-kyles-1

---

<div class="post-metadata">

**Author:** ![kyle\_che](https://avatars.discourse-cdn.com/v4/letter/k/ce73a5/32.png) [@kyle\_che](https://discuss.elastic.co/u/kyle_che)\
**Post date:** [April 11, 2019, 1:50pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/46 "2019-04-11T13:50:43Z")

</div>

maybe because i was putting the extension and hostname does not have the extension?

---

<div class="post-metadata">

**Author:** ![kyle\_che](https://avatars.discourse-cdn.com/v4/letter/k/ce73a5/32.png) [@kyle\_che](https://discuss.elastic.co/u/kyle_che)\
**Post date:** [April 11, 2019, 1:53pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/47 "2019-04-11T13:53:30Z")

</div>

do you have to be consistent? i had node.name=hostname and i had network.host=ip\_address

---

<div class="post-metadata">

**Author:** ![kyle\_che](https://avatars.discourse-cdn.com/v4/letter/k/ce73a5/32.png) [@kyle\_che](https://discuss.elastic.co/u/kyle_che)\
**Post date:** [April 11, 2019, 1:54pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/48 "2019-04-11T13:54:55Z")

</div>

this was not an issue in 6.7.1 for me

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [April 11, 2019, 1:55pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/49 "2019-04-11T13:55:02Z")

</div>

As I said above:

> [@DavidTurner](#):
>
> The exact strings listed here...
> 
> ```auto
> must discover master-eligible nodes [d-gp2-es46-1., d-gp2-es46-2., d-gp2-es46-3.]
> ^^^^^^^^^^^^^ ^^^^^^^^^^^^^ ^^^^^^^^^^^^^
> 
> ```
> 
> ... must match the strings here in between the first set of braces ...
> 
> ```auto
> have discovered [{d-gp2-es46-2}{H2bib1wCSBKGu_Ku_4DgjA}{rzokY9nmRDCBNz0lBMgUYw}{<.....>.166.183}{<.....>.166.183:9300}{ml.machine_memory=4143783936, ml.max_open_jobs=20, xpack.installed=true}
> ,{d-gp2-es46-3}{8KNzmk5uS2mZSZiftNVTDQ}{exvQChr7RPyDlkuJ-FT2Rg}{<.....>.165.141}{<.....>.165.141:9300}{ml.machine_memory=4143783936, ml.max_open_jobs=20, xpack.installed=true}]
> ^^^^^^^^^^^^
> 
> ```

No extensions or anything, they need to be exactly the same.

---

<div class="post-metadata">

**Author:** ![kyle\_che](https://avatars.discourse-cdn.com/v4/letter/k/ce73a5/32.png) [@kyle\_che](https://discuss.elastic.co/u/kyle_che)\
**Post date:** [April 11, 2019, 2:00pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/50 "2019-04-11T14:00:19Z")

</div>

ok, HOSTNAME gives me the shortname ... so i believe the problem is that i was specifying hostname with the extension and it was comparing the two and failing. i just went in and put the {HOSTNAME} back and removed the extension everywhere else and it works still. i will try the upgrade again and let you know my findings.

---

<div class="post-metadata">

**Author:** ![kyle\_che](https://avatars.discourse-cdn.com/v4/letter/k/ce73a5/32.png) [@kyle\_che](https://discuss.elastic.co/u/kyle_che)\
**Post date:** [April 11, 2019, 3:24pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/51 "2019-04-11T15:24:05Z")

</div>

yes, that was the issue ... they have to be identical with the node\_name.blah.com or just node\_name but seems that elasticsearch creates a hashcode on the node\_name so they must be identical. thanks for your help.

---

<div class="post-metadata">

**Author:** ![safderali5](https://avatars.discourse-cdn.com/v4/letter/s/8edcca/32.png) [@safderali5](https://discuss.elastic.co/u/safderali5)\
**Post date:** [April 12, 2019, 5:49am UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/52 "2019-04-12T05:49:01Z")

</div>

I tried with your recmendation, then it poped up with this error,

org.elasticsearch.transport.RemoteTransportException: [elasticsearch-data-2][172.20.13.119:9300][internal:cluster/coordination/join/validate]  
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: join validation on cluster state with a different cluster uuid KkR7myqKQU6x02Qzd5KMIw than local cluster uuid U5B7kS8rQ12gS2sJeFfbIA, rejecting

at org.elasticsearch.cluster.coordination.JoinHelper.lambda$new$4(JoinHelper.java:147) ~[elasticsearch-7.0.0.jar:7.0.0]

at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler$1.doRun(SecurityServerTransportInterceptor.java:251) ~[?:?]

at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.0.0.jar:7.0.0]

at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.messageReceived(SecurityServerTransportInterceptor.java:309) ~[?:?]

at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:63) ~[elasticsearch-7.0.0.jar:7.0.0]

at org.elasticsearch.transport.TcpTransport$RequestHandler.doRun(TcpTransport.java:1077) ~[elasticsearch-7.0.0.jar:7.0.0]

at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:751) ~[elasticsearch-7.0.0.jar:7.0.0]

at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.0.0.jar:7.0.0]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0\_202]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0\_202]

at java.lang.Thread.run(Thread.java:748) [?:1.8.0\_202]

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [April 12, 2019, 6:12am UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/53 "2019-04-12T06:12:59Z")

</div>

@safderali5 would you open another thread about your issue(s) - this thread is marked as resolved, and the problems you're facing are different from the ones we dug into here.

---

<div class="post-metadata">

**Author:** ![safderali5](https://avatars.discourse-cdn.com/v4/letter/s/8edcca/32.png) [@safderali5](https://discuss.elastic.co/u/safderali5)\
**Post date:** [April 12, 2019, 6:17am UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/54 "2019-04-12T06:17:34Z")

</div>

OK, I will it in a different thread.

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [April 12, 2019, 1:12pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/55 "2019-04-12T13:12:57Z")

</div>

We have added a note to [the docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery-bootstrap-cluster.html) to clarify this point lest it catch anyone else out:

> The node names used in this list must exactly match the `node.name` properties of the nodes. By default the node name is set to the machine’s hostname which may or may not be fully-qualified depending on your system configuration. If each node name is a fully-qualified domain name such as `master-a.example.com` then you must use fully-qualified domain names in the `cluster.initial_master_nodes` list too; conversely if your node names are bare hostnames (without the `.example.com` suffix) then you must use bare hostnames in the `cluster.initial_master_nodes` list. If you use a mix of fully-qualifed and bare hostnames, or there is some other mismatch between `node.name` and `cluster.initial_master_nodes` , then the cluster will not form successfully and you will see log messages like the following.
> 
> ` [master-a.example.com] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [master-a, master-b] to bootstrap a cluster: have discovered [{master-b.example.com}{...`
> 
> This message shows the node names `master-a.example.com` and `master-b.example.com` as well as the `cluster.initial_master_nodes` entries `master-a` and `master-b` , and it is apparent that they do not match exactly.

---

<div class="post-metadata">

**Author:** ![jswid](https://avatars.discourse-cdn.com/v4/letter/j/da6949/32.png) [@jswid](https://discuss.elastic.co/u/jswid)\
**Post date:** [April 12, 2019, 7:48pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/56 "2019-04-12T19:48:06Z")

</div>

I am also running on Kubernetes and had an issue bootstraping the cluster in 7. My solution was to set an environment variable:  
`- name: cluster.initial_master_nodes valueFrom: fieldRef: fieldPath: metadata.name`

I'm not sure this is the best solution, ideally i'd just configure the cluster to require 2/3 masters at any time like in the old ES. Using this, when the cluster first starts, the first node will already consider itself ready, and there is potential the clusters are partitioned if they don't find the other masters immediately. Hopefully it will only matter once, but I'm not really sure things will work out when updating the cluster yet.

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [April 13, 2019, 7:45am UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/57 "2019-04-13T07:45:55Z")

</div>

@jswid your formatting was mangled, but assuming you mean the following:

> [@jswid](#):
>
> ```auto
> - name: cluster.initial_master_nodes
> valueFrom:
> fieldRef:
> fieldPath: metadata.name
> 
> ```

This is not recommended. From [the docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery-bootstrap-cluster.html):

> You must set `cluster.initial_master_nodes` to the same list of nodes on each node on which it is set in order to be sure that only a single cluster forms during bootstrapping and therefore to avoid the risk of data loss.

With your suggestion you are configuring `cluster.initial_master_nodes` differently on each node, and there is a good chance that you will form more than one cluster.

---

<div class="post-metadata">

**Author:** ![jswid](https://avatars.discourse-cdn.com/v4/letter/j/da6949/32.png) [@jswid](https://discuss.elastic.co/u/jswid)\
**Post date:** [April 15, 2019, 4:21pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/58 "2019-04-15T16:21:33Z")

</div>

Yes, thanks.. i even tried to delete my comment, but I guess it didn't take. I ended up doing it a different way. I changed the Deployment to a StatefulSet, which I think is better for two reasons: one is that the node names are constant, which solves the big issue in this thread, but the other is that the masters seem to care more about a cluster uuid now, so I am now mounting a persistent volume so the nodes' `data` folders are no longer lost when the masters are updated.

I have a public template on github that more or less shows how I am planning on moving to ES7 on Kubernetes here: [https://github.com/jswidler/elasticsearch-kubed/blob/master/templates/2\_elasticsearch/es-master.yml](https://github.com/jswidler/elasticsearch-kubed/blob/master/templates/2_elasticsearch/es-master.yml)

---

<div class="post-metadata">

**Author:** ![Ken\_Liao1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ken_liao1/32/45511_2.png) [@Ken\_Liao1](https://discuss.elastic.co/u/Ken_Liao1)\
**Post date:** [May 3, 2019, 4:50pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/59 "2019-05-03T16:50:43Z")

</div>

'm trying to setup auto scaling elasticsearch cluster environment in docker swarm with this page: [http://derpturkey.com/elasticsearch-cluster-with-docker-engine-swarm-mode/](http://derpturkey.com/elasticsearch-cluster-with-docker-engine-swarm-mode/)

```auto
version: '3'  
services:  
  elasticsearch:
    image: 'elasticsearch:5'
    command: [elasticsearch, -E, network.host=0.0.0.0, -E, discovery.zen.ping.unicast.hosts=elasticsearch, -E, discovery.zen.minimum_master_nodes=1]    
    volumes:
      - /elasticsearch/data:/usr/share/elasticsearch/data
    deploy:
      mode: 'global'
      placement:
        constraints: [node.labels.app_role == elasticsearch]

```

it will automatically deploy elasticsearch node to all hosts with label: app\_role == elasticsearch;  
it works fine with elasticsearch 6.7.0, but it got the same error if I upgrade it to 7.0.0;  
The container name is dynamically changed, then what should I put for `cluster.initial_master_nodes` to fix it?

Thanks!

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [May 3, 2019, 5:09pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304/60 "2019-05-03T17:09:40Z")

</div>

Hi @Ken_Liao1, it's probably best to start a new thread with your question rather than adding it to the bottom of this (rather long) one.

[Previous page](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304.md?page=2)

[Next page](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped-v7-cluster/176304.md?page=4)