Index size questions

Elastic Stack Elasticsearch
1

All,

I am getting up to speed with the ELK stack in general and am curious about what I should expect for index sizing. I have a 3 node ES cluster and plan to use: 3 shards and 2 replicas for each index.

My first test is to ingest IIS data ( which if you are a regular to discuss.elastic you may have already read in other sections :wink: ) into ES. Using Filebeat -> Logstash -> ES, I am looking to ingest about the last 2 months worth of logs from our IIS log directory (using ignore_older) that contains daily files dating back to Jan 2016. Each IIS log is about 250MB so, I guess my math is:

Initial ingestion:

250MB x 60 days = 15,000MB = 15G

With 3 shards, 2 replicas = ~45GB per node

  1. Is this a reasonable assumption or is there other 'overhead' that will cause for more disk consumption?
  2. Is creating a single index the same as creating multiple indices (say daily with an added date to index name) disk space wise?

Any guidance is greatly appreciated.

Thanks,

HB

2

The size the data will take up on disk will depend on your mappings and how you enrich your data, as described in this blog post.

3

Christian - Thank you very much, I will read your post..

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.