Shard sizing as per the RAM available for an instance

Elastic Stack Elasticsearch
1

How do the shard size related to the RAM capacity of the es instance? From the articles I read, 20 - 25 shards per GB is ideal and each shard can be from few GB to 10s of GB but should be less than 50GB. Also, the recommended heap size for a single es instance is less than 32GB to benefit from c-oops and lesser GC time.

Now I have a 30GB heap with 8 cores and disk space of 256GB SSD. How many shards can I fit on this machine if I have 15GB of shards? Does shards completely reside in memory or will be loaded into memory as and when needed?

2

You normally don't want to have more than 20 shards per gb of HEAP.
Which means in your case 16gb of heap so no more than 320 shards on this machine.

The shard capacity should not exceed 20 to 50gb for each shard.

May I suggest you look at the following resources about sizing:

And Using Rally to Get Your Elasticsearch Cluster Size Right | Elastic Videos

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.