# Cluster.initial\_master\_nodes for ElasticSearch on EC2 with Auto Scaling **URL:** \ **Category:** Elasticsearch\ **Created:** [August 14, 2023, 10:20pm UTC](https://discuss.elastic.co/t/cluster-initial-master-nodes-for-elasticsearch-on-ec2-with-auto-scaling/340784 "2023-08-14T22:20:14Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![cockroachmondays](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cockroachmondays/32/124563_2.png) [@cockroachmondays](https://discuss.elastic.co/u/cockroachmondays)\ **Post date:** [August 14, 2023, 10:20pm UTC](https://discuss.elastic.co/t/cluster-initial-master-nodes-for-elasticsearch-on-ec2-with-auto-scaling/340784/1 "2023-08-14T22:20:14Z")
Trying to upgrade ES from 6.8 to 8.9 for ES running on EC2 AWS instances. The issue is within cluster.initial\_master\_nodes. I see that I set fixed values for node.name. However, the instances are created by Auto Scaling group (Terraform configs), the ES template is one per group and I don't see the possibility of passing the index count of the instance created. We need a scaling group to ensure there’s always the same number of instances at any given time. And looks like there’s no other way to achieve that. Looks like there's no way to use ES 7+ with AWS Scaling group. Previously, 'cluster.initial\_master\_nodes' wasn't needed and node.name was just set to ${HOSTNAME}. Any ideas?.. ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [August 15, 2023, 4:32pm UTC](https://discuss.elastic.co/t/cluster-initial-master-nodes-for-elasticsearch-on-ec2-with-auto-scaling/340784/2 "2023-08-15T16:32:53Z")
You only need `cluster.initial_master_nodes` the first time the cluster starts. You can get this to work by starting a "seed" node outside the ASG, with fixed node name and `cluster.initial_master_nodes`, then starting up the rest of the cluster and having it join the seed node, and then finally shutting down the seed node. However note that auto-scaling groups are designed more for stateless services, they don't work particularly well with stateful services like ES. ---
**Author:** ![cockroachmondays](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cockroachmondays/32/124563_2.png) [@cockroachmondays](https://discuss.elastic.co/u/cockroachmondays)\ **Post date:** [August 15, 2023, 11:09pm UTC](https://discuss.elastic.co/t/cluster-initial-master-nodes-for-elasticsearch-on-ec2-with-auto-scaling/340784/3 "2023-08-15T23:09:50Z")
sorry, this looks like a big hack. still looking for solutions... ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [August 16, 2023, 12:30am UTC](https://discuss.elastic.co/t/cluster-initial-master-nodes-for-elasticsearch-on-ec2-with-auto-scaling/340784/4 "2023-08-16T00:30:54Z")
Can you provide a little more context of what you are trying to do? What you want to auto scale? Master nodes? Data nodes? Ingest nodes? And what would trigger the scale-out? Would you scale-in as well? I don't think that auto scale would work well for Elasticsearch, when you add a new node the cluster will start rebalancing the shards, which can take a some time depending on the size and number of shards, and to safely scale-in you would need to empty the node before removing it, and this also can take some time. Can you give an example of how you would use auto-scaling with ES and what this would solve? Another question, are you upgrading or creating a new cluster? If you are upgrading you would not need to use `cluster.initial_master_nodes` as a cluster was already formed, also, not sure what path you are using to upgrade, but you can't go from 6.8 directly to 8.9, you need to go to 7.17 first. > [@cockroachmondays](#): > > However, the instances are created by Auto Scaling group (Terraform configs), the ES template is one per group and I don't see the possibility of passing the index count of the instance created. Also, it is not clear how this is related to auto scaling and what you mean by index count. Are you talking about number of shards and replicas? ---
**Author:** ![cockroachmondays](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cockroachmondays/32/124563_2.png) [@cockroachmondays](https://discuss.elastic.co/u/cockroachmondays)\ **Post date:** [August 16, 2023, 12:42am UTC](https://discuss.elastic.co/t/cluster-initial-master-nodes-for-elasticsearch-on-ec2-with-auto-scaling/340784/5 "2023-08-16T00:42:57Z")
Autoscaling group is used only for the restart - I want the failed nodes to be created again. Nothing is really scaled there. I have the same strategy for master and data nodes. My upgrade is strategy is to destroy the cluster and start a new one. This ES is used for logs. ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [August 16, 2023, 4:51am UTC](https://discuss.elastic.co/t/cluster-initial-master-nodes-for-elasticsearch-on-ec2-with-auto-scaling/340784/6 "2023-08-16T04:51:04Z")
> [@cockroachmondays](#): > > I want the failed nodes to be created again. This works fine for stateless services, but unfortunately you can't rely on it for stateful systems like ES. If two nodes fail at around the same time, and those are the two of the three with the latest cluster state, then creating two new nodes won't bring the cluster back. This might have _seemed_ to work in 6.8 and earlier, but in fact that was a bug that could lead to data loss and was fixed in 7.0. Unfortunately there is no way to do this that doesn't "look like a big hack" 😉 ASGs don't do what you need them to here. ---
**Author:** ![cockroachmondays](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cockroachmondays/32/124563_2.png) [@cockroachmondays](https://discuss.elastic.co/u/cockroachmondays)\ **Post date:** [August 16, 2023, 10:56pm UTC](https://discuss.elastic.co/t/cluster-initial-master-nodes-for-elasticsearch-on-ec2-with-auto-scaling/340784/7 "2023-08-16T22:56:12Z")
Thank you for the information. " This might have _seemed_ to work in 6.8 and earlier, but in fact that was a bug that could lead to data loss and was fixed in 7.0." 😅 Alright, we decided to leave autoscaling with the 6.8, moving to the new chapter without it. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 13, 2023, 10:56pm UTC](https://discuss.elastic.co/t/cluster-initial-master-nodes-for-elasticsearch-on-ec2-with-auto-scaling/340784/8 "2023-09-13T22:56:19Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.