Thanks @Christian_Dahlqvist for your time and the benchmarking document.
I agree for the consideration of primary and replica shard for the per heap GB calculation. It's a miss from me.
ES version I use is 7.17.
My initial calculation was considering the max. shard possible for node is in my case 560 (28GB heap) and max. shard size 50GB, I know that's extreme, all put-up for the calculation.
I assumed all those data (per GB restriction and max. shard size, captured from elastic.co) mostly by considering the query load and indexing data may be much more. Also I left 50% of RAM for OS caching.
But I am bit confused with the calculation in the mentioned blog.
The typical memory:data ratio for the hot zone used is 1:30 and for the warm zone is 1:160.
1:30 memory disk ratio for hot data node limits max. 2TB disk per node,
If I go with my previous calculation considering 40GB of single shard size.
230TB/40 = 5750 total shards.
5750/500 shards per node = 11.5 ~ 12 data nodes. and 560*40GB = 20TB far higher than 2TB size.
But considering hot and warm node architecture and memory-disk ratio,
Hot zone = 7 days.
Warm zone = 83 days.
Hot zone
Total data (GB) = 85 * 15 * 7 * (1+1) = 17850GB
Total Storage (GB) = 17850 * 1.25 =22313GB
Total Data Nodes = ROUNDUP(22313/64/30)+1 = 13 nodes.
Warm zone
Total data (GB) = 85 * 15 * 83 * (1+1) = 211650GB
Total Storage (GB) = 211650 * 1.25 =264563GB
Total Data Nodes = ROUNDUP(264563/64/160)+1 = 27 nodes.
Total of 40 data nodes.
- But based on this calculation either a single shard size or shard per GB is presumably low.
- Hope I can still decide the no. of shards based on the current day's index size.,
e.g., consider 40GB per shard, 85GB index size => 85/40 = 3 + 1 replica for each shard. - I am now fallen into two of the above approach, could you please suggest on this.
My stack is basically ingest logs from various sources and access from Kibana for tracing and tracking. I do keep a couple of dedicated coordinating nodes and 3 master nodes. All runs as on-Prem VMs.
Found the similar thread Clarification about recommended memory-disk ratio of 1:30 - #2 by DavidTurner