ElasticSearch Index / Shard Setting

Elastic Stack Elasticsearch
1

I am trying to create a multi node cluster using 2 machines. below is what i am trying to do.

Master 1 (master=true,data = false)
data 1 (data=true,master=false)
data 2 (data=true,master=false)

I want to retain the default shard/replica settings of index.number_of_shards: 5 and index.number_of_replicas: 1. i have 2 Questions,

  1. Where should the shard and replica setting be specified in the master node config or data node config?
  2. Also if in future if i add more nodes (master/data) w.r.t index/replica anything i need to do ?

Thanks

2

You ideally want to have 3 master eligible nodes in the cluster, so I would recommend making all 3 nodes master eligible. Running with a single dedicated master node makes it a single point of failure. Once you have 3 master eligible nodes, you need to set minimum_master_nodes to 2 (instead of the default 1) in order to prevent split brain scenarios.

Those are the default settings, so no need to specify tin the config. If you change it, do it on all nodes.

Even for larger clusters you can keep 3 nodes master eligible, which means that you do not need to update any settings. If you however increase the number of master eligible nodes, you must update 'minimum_master_nodes' to be (Number of master eligible nodes/2)+1 (a majority of your master eligible nodes).

3

Thanks !!!
makes sense. I will test this out.

4

Thanks the nodes are setup and able to communicate with each other. I am able to send data to one of the datanode through logstash.

I will try out couple of things I have in mind and will create a separate topic if I have doubts.