# Master not discovered yet, this node has not previously joined a bootstrapped **URL:** \ **Category:** Elasticsearch\ **Created:** [November 12, 2021, 2:30pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped/289075 "2021-11-12T14:30:36Z")\ **Posts on this page:** 17\ **Page:** 1
**Author:** ![Maks\_7v](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maks_7v/32/97067_2.png) [@Maks\_7v](https://discuss.elastic.co/u/Maks_7v)\ **Post date:** [November 12, 2021, 2:30pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped/289075/1 "2021-11-12T14:30:36Z")
Hi there. I'm trying to deploy 3 masters, 2 data, and 2 ingest nodes to GCP k8s. There is a problem that masters struggle to discover each other. Here are the logs I get: `"master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [es-master-0, es-master-1, es-master-2] to bootstrap a cluster: have discovered [{es-master-0}{SPyxwBwVQJ2SVlsI2pLZFg}{62jGZWMZR_KxyGx-OQSC3A}{10.84.0.141}{10.84.0.141:9300}{m}]; discovery will continue using [127.0.0.1:9300, 127.0.0.1:9301, 127.0.0.1:9302, 127.0.0.1:9303, 127.0.0.1:9304, 127.0.0.1:9305] from hosts providers and [{es-master-0}{SPyxwBwVQJ2SVlsI2pLZFg}{62jGZWMZR_KxyGx-OQSC3A}{10.84.0.141}{10.84.0.141:9300}{m}] from last-known cluster state; node term 0, last-accepted version 0 in term 0"` The weird thing is that es-master-0 discovers only es-master-0, and the same about other master pods. Here is my Elasticsearch.yaml: ```auto elasticsearch.yml: |- cluster.name: $ES_CLUSTER_NAME cluster.initial_master_nodes: es-master-0,es-master-1,es-master-2 network.host: 0.0.0.0 node.roles: master xpack.security.enabled: true xpack.monitoring.collection.enabled: true ``` ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [November 12, 2021, 4:05pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped/289075/2 "2021-11-12T16:05:54Z")
You need to tell Elasticsearch how to discover the other master nodes, typically by setting `discovery.seed_hosts`. By default discovery will only try a few ports on localhost: `discovery will continue using [127.0.0.1:9300, 127.0.0.1:9301, 127.0.0.1:9302, 127.0.0.1:9303, 127.0.0.1:9304, 127.0.0.1:9305] from hosts providers` ---
**Author:** ![Maks\_7v](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maks_7v/32/97067_2.png) [@Maks\_7v](https://discuss.elastic.co/u/Maks_7v)\ **Post date:** [November 12, 2021, 4:38pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped/289075/3 "2021-11-12T16:38:27Z")
Thanks for a quick reply, David! Added `discovery.seed_hosts` the same way as `cluster.initial_master_nodes`: The previous error disappeared, but there is a new problem: `{"type": "server", "timestamp": "2021-11-12T16:33:35,960Z", "level": "WARN", "component": "o.e.d.SeedHostsResolver", "cluster.name": "es-backup-cluster", "node.name": "es-master-0", "message": "failed to resolve host [es-master-1]"` `"java.net.UnknownHostException: es-master-1: Name or service not known"` ---
**Author:** ![ahmed\_charafouddine](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ahmed_charafouddine/32/45129_2.png) [@ahmed\_charafouddine](https://discuss.elastic.co/u/ahmed_charafouddine)\ **Post date:** [November 12, 2021, 4:49pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped/289075/4 "2021-11-12T16:49:24Z")
It seems he has a problem with DNS resolve ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [November 12, 2021, 4:51pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped/289075/5 "2021-11-12T16:51:07Z")
Yes as @ahmed_charafouddine says: Elasticsearch is doing a DNS lookup for `es-master-1` but your DNS doesn't know this name. Either add this hostname to DNS or else just use its address. ---
**Author:** ![Maks\_7v](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maks_7v/32/97067_2.png) [@Maks\_7v](https://discuss.elastic.co/u/Maks_7v)\ **Post date:** [November 12, 2021, 5:58pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped/289075/6 "2021-11-12T17:58:41Z")
Ok, got it. Now I'm trying to use pod's address. Can you help with this, please? I try to check the connection to one pod from another using this pattern: `pod-name.service-name.namespace.svc.cluster.local` in my case, it is: `es-master-2.es-master.es-cluster.svc.cluster.local` and I cannot access it. Am I doing it wrong? Between, I can connect to the service like this: `nc es-master.es-cluster.svc.cluster.local` ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [November 12, 2021, 6:33pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped/289075/7 "2021-11-12T18:33:08Z")
I'm not sure, this sounds like a networking-and-containers question rather than anything specific to Elasticsearch, so this might not be the best place to ask. ---
**Author:** ![Maks\_7v](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maks_7v/32/97067_2.png) [@Maks\_7v](https://discuss.elastic.co/u/Maks_7v)\ **Post date:** [November 12, 2021, 7:02pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped/289075/8 "2021-11-12T19:02:21Z")
I've found a way to connect from one master k8s pod to another here is an address that I use: `es-master-2.es-master.es-cluster.svc.cluster.local` I've added such addresses to `discovery.seed_hosts` and now I have the same error that was before: `master not discovered yet, this node has not previously joined` here is my current Elasticsearch.yaml: ```auto elasticsearch.yml: |- cluster.name: $ES_CLUSTER_NAME discovery.seed_hosts: - es-master-0.es-master.es-cluster.svc.cluster.local - es-master-1.es-master.es-cluster.svc.cluster.local - es-master-2.es-master.es-cluster.svc.cluster.local network.host: 0.0.0.0 node.roles: master xpack.security.enabled: true xpack.monitoring.collection.enabled: true ``` ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [November 12, 2021, 7:09pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped/289075/9 "2021-11-12T19:09:55Z")
I think the message won't be exactly the same, and the differences are important. What is the full message? ---
**Author:** ![Maks\_7v](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maks_7v/32/97067_2.png) [@Maks\_7v](https://discuss.elastic.co/u/Maks_7v)\ **Post date:** [November 12, 2021, 7:15pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped/289075/10 "2021-11-12T19:15:09Z")
`"master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and [cluster.initial_master_nodes] is empty on this node: have discovered [{es-master-0}{vUoyCU2sQ5OFnXv9Bj-LHA}{ri7xpeDlSuOclHnkuiXvbg}{10.84.0.150}{10.84.0.150:9300}{m}, {es-master-1}{CpLNUEKlQP6ZSHtL-6LjEQ}{AG17cWBqS7aZlQSXUuAtqw}{10.84.2.119}{10.84.2.119:9300}{m}, {es-master-2}{b4DtEy9lTnGVCRm-RareKg}{344M5bIrSwGbNsL0zYgaNA}{10.84.3.118}{10.84.3.118:9300}{m}]; discovery will continue using [10.84.2.119:9300, 10.84.3.118:9300] from hosts providers and [{es-master-0}{vUoyCU2sQ5OFnXv9Bj-LHA}{ri7xpeDlSuOclHnkuiXvbg}{10.84.0.150}{10.84.0.150:9300}{m}] from last-known cluster state; node term 0, last-accepted version 0 in term 0"` ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [November 12, 2021, 8:02pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped/289075/11 "2021-11-12T20:02:39Z")
> [@Maks\_7v](#): > > `discovery will continue using [10.84.2.119:9300, 10.84.3.118:9300] from hosts providers` Are these addresses correct? Are there any other messages in the logs? What version are you using? I think in recent versions it'll be logging more details too. ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [November 12, 2021, 8:03pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped/289075/12 "2021-11-12T20:03:57Z")
Oh wait, yes the message is completely different: > [@Maks\_7v](#): > > `[cluster.initial_master_nodes] is empty on this node` You need to set `cluster.initial_master_nodes` too (but only the first time the cluster forms) ---
**Author:** ![Maks\_7v](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maks_7v/32/97067_2.png) [@Maks\_7v](https://discuss.elastic.co/u/Maks_7v)\ **Post date:** [November 12, 2021, 8:53pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped/289075/13 "2021-11-12T20:53:54Z")
added initial\_master\_nodes and still similar issue as the first one: master-0 discovers only master-0, and there same for other pods `"master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [es-master-0.es-master.es-cluster.svc.cluster.local, es-master-1.es-master.es-cluster.svc.cluster.local, es-master-2.es-master.es-cluster.svc.cluster.local] to bootstrap a cluster: have discovered [{es-master-0}{wJc_UcmwTVaQhL0TBOS93A}{yf9TrRd7QUWTWq77Lncdow}{10.84.0.151}{10.84.0.151:9300}{m}, {es-master-1}{tfghLpmVQLmNN2H_O_ituw}{VRlYo2PvR2-GXkS0YvmzmQ}{10.84.2.120}{10.84.2.120:9300}{m}, {es-master-2}{-jP3pAubQ8i1WMIYCGHjoQ}{oJ3Z1nfHR2CZ3CSYZwE58w}{10.84.3.119}{10.84.3.119:9300}{m}]; discovery will continue using [10.84.2.120:9300, 10.84.3.119:9300] from hosts providers and [{es-master-0}{wJc_UcmwTVaQhL0TBOS93A}{yf9TrRd7QUWTWq77Lncdow}{10.84.0.151}{10.84.0.151:9300}{m}] from last-known cluster state; node term 0, last-accepted version 0 in term 0"` ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [November 12, 2021, 10:14pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped/289075/14 "2021-11-12T22:14:01Z")
`cluster.initial_master_nodes` should be set to the node names (`es-master-0`, etc), not their fully-qualified DNS names. The setup in your first post was right. See [these docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/important-settings.html#initial_master_nodes) for more information. ---
**Author:** ![Maks\_7v](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maks_7v/32/97067_2.png) [@Maks\_7v](https://discuss.elastic.co/u/Maks_7v)\ **Post date:** [November 15, 2021, 1:38pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped/289075/15 "2021-11-15T13:38:54Z")
Thank you David, now all the k8s pods see each other, so issue could be closed For other users I will share my final config, maybe it will be helpful: ```auto elasticsearch.yml: |- cluster.name: $ES_CLUSTER_NAME discovery.seed_hosts: - es-master-0.es-master.es-cluster.svc.cluster.local - es-master-1.es-master.es-cluster.svc.cluster.local - es-master-2.es-master.es-cluster.svc.cluster.local cluster.initial_master_nodes: - es-master-0 - es-master-1 - es-master-2 network.host: 0.0.0.0 node.roles: master xpack.security.enabled: true xpack.monitoring.collection.enabled: true ``` In my set up I use StatefulSet for master and data nodes All the services have both cluster.initial\_master\_nodes and discovery.seed\_hosts configured as shown above `es-master-0.es-master.es-cluster.svc.cluster.local` is - `podName.serviceName.namespace.svc.cluster.local` if there is no namespace specified - put "default" there ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [November 15, 2021, 2:29pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped/289075/16 "2021-11-15T14:29:59Z")
That's great, thanks for confirming it all works now. > [@Maks\_7v](#): > > For other users I will share my final config, maybe it will be helpful: I think this shouldn't be your final config, you should remove `cluster.initial_master_nodes` after the first time the cluster forms. See [these docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/important-settings.html#initial_master_nodes) for more info. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 13, 2021, 2:30pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped/289075/17 "2021-12-13T14:30:10Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.