We have requirement to build ES cluster for 10TB per day to retain the data for 60 days.
With the configuration of Replication shard to 2. Primary shard may be depending on the index receiving the daily volume which can vary.
what is the data storage required for 10TB data raw data with replica set 2?
Is it 10TB (primary shard) + 2 * TB (for replica shards) = 30TB the storage required in the ES cluster?
Is there is any compression applied for shards? If yes, what will be space required for 10TB raw data (log files)?
What is the best number of primary shard for the index which gets 1TB per day?
If there is any formula to calculate the storage space required, please let me know.
Planning to build ES node with 32GB RAM, 18 CPU cores. What should be shard size for the better performance?