Elasticsearch - question about shards

Elastic Stack Elasticsearch
1

Hi there!

We have 2 ES instances in our environment with 2 daily index and we already have more than 7k shards.
It is normal?
If not, what are you suggest?

/_cat/health?v
epoch      timestamp cluster       status node.total node.data shards  pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1527602653 16:04:13  logs green           2         2   7760 3880    0    0        0             0                  -                100.0%
2

You probably have too many shards per node.

May I suggest you look at the following resources about sizing:

https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing

1 Like
3

Thank you for your links.
I figured out that our shards size is very small:

GET _cat/shards/logs-2018.05.29
logs-2018.05.29 1 r STARTED 196755 176.9mb 192.168.2.161 db1
logs-2018.05.29 1 p STARTED 196755 176.7mb 192.168.2.162 db2
logs-2018.05.29 3 r STARTED 197144 176.9mb 192.168.2.161 db1
logs-2018.05.29 3 p STARTED 197144 176.6mb 192.168.2.162 db2
logs-2018.05.29 4 p STARTED 196554 176.2mb 192.168.2.161 db1
logs-2018.05.29 4 r STARTED 196554 177.4mb 192.168.2.162 db2
logs-2018.05.29 2 p STARTED 196979 177.8mb 192.168.2.161 db1
logs-2018.05.29 2 r STARTED 196979 176.7mb 192.168.2.162 db2
logs-2018.05.29 0 p STARTED 196328 176.1mb 192.168.2.161 db1
logs-2018.05.29 0 r STARTED 196328 176.5mb 192.168.2.162 db2

As I understand, it is common to have shard size around 30GB. Currently I have number_of_shards set to 5 and this is probably my problem. I will set this parameter to 2.

4

Even better, set it to 1 :slight_smile:

1 Like
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.