Shard size issue in elastic

[Muna]

How to configure the optimal shard size in elastic search?

I've gone through multiple posts in this forum and I'm not able to find an exact way to calculate the optimal value for cluster.max_shards_per_node

Increasing this max_shards_per_node as temporary fix is working but as more index gets logged into elastic, the same issues occurs.

For instance say we have 1000 indices, 5 shards per node and 50gb disk space, what would be the optimal value for cluster.max_shards_per_node ?

[Christian_Dahlqvist]

Aim to have shards with an average size of tens of GB. If you only have 50GB storage per node you need single digit shards per node. There is no need to have 5 replicas per index and fewer indices would be better.

[Muna]

Thanks Christian for the suggestion, could you please let me know an approach for setting value to total shards per node?

[Christian_Dahlqvist]

You should not override this. It seems you already have though so I would recommend fixing your sharping issue and then set it back to the default value. Further increasing this will most likely just let you get to a situation where you no longer can fix your cluster as cluster updates time out and lose potentially lose all your data as you have to start over from scratch.