When it comes to how much data you can hold on a node it often comes down to how well you manage heap usage, as this is likely to be the limiting factor. It can therefore vary from use-case to use case.
With a 15GB heap I think you are unlikely to get to 10TB, which means that you either end up with smaller or fewer shards.
A good rule-of-thumb is to ensure you keep the number of shards per node below 20 to 25 per GB heap it has configured
So 15GB Heap size multiplied by minimum of 20 shards means I can have minimum 300 shards in a node with 30GB RAM
Aim to keep the average shard size between a few GB and a few tens of GB. For use-cases with time-based data, it is common to see shards between 20GB and 40GB in size
If I use the minimum 20GB per shard multiplied by minimum number of 300 shards gives be 6TB of minimum.
And I have a replication factor of 2
It is not a minimum, it is a range for a maximum. The most common problem we see is that users end up having far too many small shards, which is why we have provided a recommendation around the maximum number of shards you should have. If you are using large shards you are likely to be below this recommended maximum number of shards per node.
A good rule-of-thumb is to ensure you keep the number of shards per node below 20 to 25 per GB heap it has configured
So 25 is maximum. That means 375 shards
Aim to keep the average shard size between a few GB and a few tens of GB. For use-cases with time-based data, it is common to see shards between 20GB and 40GB in size
40GB is maximum. So 375*40 = 15000 = 14.6TB Disk space is maximum
My shard size is 35GB currently
How can you have 11622 shards of 35GB (397TB) on 15 nodes with 2TB of disk each?
There are many shards that are very small(single shard single index)
According to the statistics, your average shard size is around 1.6GB. If you want to store data efficiently I would say this is on the small side.
The large number of small shards you have, which brings down the average size, will consume resources and affect cluster performance. I would recommend reducing and/or consolidating these, e.g. by changing the number of primary shards, consolidating indices and/or switching to weekly or monthly indices instead of daily.
Thank you so much @Christian_Dahlqvist. Your help is much appreciated 
If you are interested in getting the most from your nodes with respect to storage, we are planning a webinar in November that may be of interest.
Sure.... Thank you so much
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.