Number of Shards in Master-Worker topology

Elastic Stack Elasticsearch
1

Hi,

I was playing around with the Elastic Topology and created a topology where the master node had different number of shards than that configured in the worker node configs.

eg. Master config - index.number_of_shards:10
Worker2 config - index.number_of_shards:8
Worker2 config - index.number_of_shards:12

Can anyone explain what is the relationship between the number of shards mentioned in master and that of the workers.

What I noticed is that, the cluster is always taking the number of shards mentioned in the master, no matter what numbers are mentioned in the worker.

Does is mean that I can't have different number of shards in different nodes depending upon my requirement ?
Meaning, if I know that for a particular Node the data (index in that node) is such that creating more shards and routing queries would be more helpful but for another node I want only less, say, only 1 shard.

Any insights would be of great help.

Thanks
Shovik

2

This setting defines the default number of shards to use when a new index is created. It does not affect the distribution or allocation of shards across the nodes, so it makes no sense to set if to different values of different nodes.

3

Like Christian said this is the setting to be used while creating a new index. ES by default does distribution of shards to nodes. In your scenario u can think of multiple indices and use include/exclude tags to put specific indices in specific nodes.