Max number of shard a elastic node can hold

Elastic Stack Elasticsearch
1

Hi all
What is the max number of shard that an elastic node can hold.
and i dont mean per index or anything i mean the max number of shard for all index that a node can hold.

2

The default limit is set to 1000 but you should generally aim to be significantly below that if you can.

3

Thanks for your replies,
Is there a way to increase this limit.

Is there a reason for this recommendation

4

Too many shards puts pressure on the JVM and can cause overload.

5

Having lots of small shards is inefficient and can cause performance problems. In addition to affecting heap usage due to overhead per shard it can also cause the cluster state to be large and as a consequence be slow to update which can cause instability.

How many shards are looking to have? What would the average size be? What is the driver behind this large number of shards?

6

Currently in my system there is about 6 type of index that are daily indices and we have to store them for 6 month data so it is about close to 3500 shard for my system of 4 data node.

The size of each shard are quite small to about few dozen GB.

I could go to look into using index life cycle but our system is too far gone to setup the life cycle now. If i knew the life cycle from the start all of this would have been much easier.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.