How to identify number of shards based on the nodes

Elastic Stack Elasticsearch
1

I have a 5 node Elasticsearch cluster of version 5.2.1. It consist of 3 master and 2 data node. I defined no.of shards as 7 and replica as 1.

With this setting sometimes my shards health are going to yellow status. Which causing Elasticsearch timeout issue when I access Kibana I doubt it whether the number of shards causing this issue.

Is there any formula to identify number of shards based on the nodes in the cluster? . Please provide your insights.

2

How many shards do you have in the cluster? What is the average shard size?

As you have only 2 data nodes (the only nodes that hold shards) both nodes will hold a copy of all data as you have 1 replica configured. I would recommend reading this blog post for some practical guidance on shard count and size.

3

I have 7 shards and 1 replica and my avg shard size is 25GB

4

So only a single index?

5

I have 16 different application indices.

6

So 16 * 7 * 2 = 224 shards in total?

7

Yes and the avg size of 1 shard is 25GB not 130 GB. I corrected my above response

8

If you want to see how shards are distributed across the cluster, you can use the cat shards API. Is there anything in the Elasticsearch logs about the yellow status you mentioned?

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.