I am getting up to speed with the ELK stack in general and am curious about what I should expect for index sizing. I have a 3 node ES cluster and plan to use: 3 shards and 2 replicas for each index.
My first test is to ingest IIS data ( which if you are a regular to discuss.elastic you may have already read in other sections ) into ES. Using Filebeat -> Logstash -> ES, I am looking to ingest about the last 2 months worth of logs from our IIS log directory (using ignore_older) that contains daily files dating back to Jan 2016. Each IIS log is about 250MB so, I guess my math is:
250MB x 60 days = 15,000MB = 15G
With 3 shards, 2 replicas = ~45GB per node
- Is this a reasonable assumption or is there other 'overhead' that will cause for more disk consumption?
- Is creating a single index the same as creating multiple indices (say daily with an added date to index name) disk space wise?
Any guidance is greatly appreciated.