# Initial master nodes ES 7.0.0 **URL:** \ **Category:** Elasticsearch\ **Created:** [April 12, 2019, 6:31am UTC](https://discuss.elastic.co/t/initial-master-nodes-es-7-0-0/176544 "2019-04-12T06:31:12Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![safderali5](https://avatars.discourse-cdn.com/v4/letter/s/8edcca/32.png) [@safderali5](https://discuss.elastic.co/u/safderali5)\ **Post date:** [April 12, 2019, 6:31am UTC](https://discuss.elastic.co/t/initial-master-nodes-es-7-0-0/176544/1 "2019-04-12T06:31:12Z")
Does all the nodes specified in cluster.initial\_master\_nodes: must be up and running before the first election for master occurs? if yes, any ideas how to do that in kubernetes. As in kubernetes, the statefulset pods are started sequentially. ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [April 12, 2019, 6:52am UTC](https://discuss.elastic.co/t/initial-master-nodes-es-7-0-0/176544/2 "2019-04-12T06:52:36Z")
No, you don't need them all up, but you do need more than half of them to be running. By default a Kubernetes StatefulSet will start the pods up one at a time and will only start each pod once the previous one is healthy. You can alter this behaviour to avoid the probes by setting `podManagementPolicy` to `Parallel`, or else you can account for this in your readiness probe implementation: a pod can be considered ready once it responds to `GET /`. The [Helm chart](https://github.com/elastic/helm-charts/blob/31f583db6afa047d998f7f090f5727480c8dfbd4/elasticsearch/templates/statefulset.yaml#L111) implements this latter idea. ---
**Author:** ![safderali5](https://avatars.discourse-cdn.com/v4/letter/s/8edcca/32.png) [@safderali5](https://discuss.elastic.co/u/safderali5)\ **Post date:** [April 12, 2019, 7:07am UTC](https://discuss.elastic.co/t/initial-master-nodes-es-7-0-0/176544/3 "2019-04-12T07:07:14Z")
I can try setting podManagementPolicy to Parallel. See if it works with ES setup. I have tried the readiness probe specified in [helm chart](https://github.com/elastic/helm-charts/blob/31f583db6afa047d998f7f090f5727480c8dfbd4/elasticsearch/templates/statefulset.yaml#L111), I had problems when using that probe with data nodes. It was stuck in loop when one data node is upgrading and cluster state turns red. It never turns green until node is back and traffic resumes. While that will only happen if the readiness probe pass which awaits for it to be green. ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [April 12, 2019, 7:15am UTC](https://discuss.elastic.co/t/initial-master-nodes-es-7-0-0/176544/4 "2019-04-12T07:15:11Z")
> [@safderali5](#): > > one data node is upgrading and cluster state turns red I think this means you have some indices without any replicas, or else the cluster health wasn't green to start with. A properly-configured cluster should only degrade from green to yellow health when one node is shut down. However it looks like the readiness probe in the Helm chart is always waiting for green health; maybe it should consider yellow health to be acceptable too to allow for upgrades. ---
**Author:** ![safderali5](https://avatars.discourse-cdn.com/v4/letter/s/8edcca/32.png) [@safderali5](https://discuss.elastic.co/u/safderali5)\ **Post date:** [April 12, 2019, 7:32am UTC](https://discuss.elastic.co/t/initial-master-nodes-es-7-0-0/176544/5 "2019-04-12T07:32:09Z")
data nodes: 3 index: 1 shards: 3 replicas: 1 I have above mentioned settings, when i do a rolling upgrade on es data nodes, one node completely goes down, and cluster state is no more green, and readiness probe in helm chart do not tolerate that. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [April 12, 2019, 7:34am UTC](https://discuss.elastic.co/t/initial-master-nodes-es-7-0-0/176544/6 "2019-04-12T07:34:13Z")
How many of your nodes are master eligible? ---
**Author:** ![safderali5](https://avatars.discourse-cdn.com/v4/letter/s/8edcca/32.png) [@safderali5](https://discuss.elastic.co/u/safderali5)\ **Post date:** [April 12, 2019, 7:34am UTC](https://discuss.elastic.co/t/initial-master-nodes-es-7-0-0/176544/7 "2019-04-12T07:34:36Z")
3 nodes are master eligible with only master role enabled ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [April 12, 2019, 7:37am UTC](https://discuss.elastic.co/t/initial-master-nodes-es-7-0-0/176544/8 "2019-04-12T07:37:50Z")
> [@safderali5](#): > > readiness probe in helm chart do not tolerate that. Yes, I'm going to ask the team about that. It's the `green` in this line; perhaps `yellow` would be ok? > ---
**Author:** ![safderali5](https://avatars.discourse-cdn.com/v4/letter/s/8edcca/32.png) [@safderali5](https://discuss.elastic.co/u/safderali5)\ **Post date:** [April 12, 2019, 8:03am UTC](https://discuss.elastic.co/t/initial-master-nodes-es-7-0-0/176544/9 "2019-04-12T08:03:42Z")
I tried with [`podManagementPolicy` to `Parallel`](https://kubernetes.io/docs/tutorials/stateful-application/basic-stateful-set/#parallel-pod-management) It still do not work, The situation at the moment is that, I have three master nodes, they all come up and running. Each nodes elects itself as master, and I end up with three clusters with one master node per cluster. The minimum master nodes settings `discovery.zen.minimum_master_nodes: 2` also do not have any effect, it shows in logs set to -1. I also could not find this setting in docs of ES7.0.0. Maybe it is set differently now. ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [April 12, 2019, 8:21am UTC](https://discuss.elastic.co/t/initial-master-nodes-es-7-0-0/176544/10 "2019-04-12T08:21:06Z")
> [@safderali5](#): > > Each nodes elects itself as master, and I end up with three clusters with one master node per cluster This indicates that `cluster.initial_master_nodes` is set wrongly. From [the docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery-bootstrap-cluster.html): > You must set `cluster.initial_master_nodes` to the same list of nodes on each node on which it is set in order to be sure that only a single cluster forms during bootstrapping and therefore to avoid the risk of data loss. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 10, 2019, 8:21am UTC](https://discuss.elastic.co/t/initial-master-nodes-es-7-0-0/176544/11 "2019-05-10T08:21:08Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.