Handling cluster.initial_master_nodes on automated provisioning

Elastic Stack Elasticsearch
1

Hi,

I'm trying to deploy a ES Cluster in Azure using Machine Scale Sets; to allow us to increase the number of nodes quickly.I would like these to be configured automatically using packer to create the machine image prior to spinning them up.

At the moment I'm just testing locally on my sandbox environment.

I'm using CentOS 7 as the OS and ES 7.5.2.

The problem I'm facing is with the cluster.initial_master_nodes setting.

It seems that I need to enter a list of node names across all of the nodes prior to ES forming a cluster however I will not know this prior to spinning the machines up and I would like to disable SSH access to the boxes.

I'm using a DNS entry to handle discovery.seed_hosts which can be controlled outside of each node.

Any advice would be really useful as it seems I'm missing something.

Thanks!

2

Hi @LED,

I think you should aim for a setup with dedicated master nodes. The autoscaling of nodes should then only affect non-master nodes (data nodes, ingest, hot/warm tiers and so forth).

Autoscaling to add a bunch of master nodes will only cause issues.

You should use minimum 3 dedicated master nodes. There is typically no reason to use more than that.

If you also intend to downscale your cluster, you need to be careful to either vacate data nodes of data before decommissioning them or otherwise ensure you have all data available without the copies on the nods that are shut down/removed.

3

@HenningAndersen,

Thanks for your advice! I'll update my plan to provision a set number of master nodes and then look at scaling for the others.

Much appreciated!