I don't know, if the post in ES is correct or if logstash forum fits better.
As I understand, the default chard size for indexes created by logstash is 5.
Currently we have 1 index per day for all types which have about 10Gb per day. 5 shards per index.
Now I would like to redesign a bit, to use multiple indexes. That would allow us to have different retention times (time until delete) per index which holds a set of logfiles / types.
We are currently using only a single instance of ELK.
To reduce the amount of memory needed by ES, I would like to decrease the shard size per index.
Here are my questions:
- Does the memory (RAM) consumed by shards only depend on the count of shards, or does it also depend on the size of the shards / indizes?
- How can I set the shard count on index level? Can it be done via logstash with some parameter, or do I need / modify the shard count in ES? How?
- Is my understanding correct, that I can have different shard count on different indizes? Can I change the chard count on the next index rotation without the need of reindexing old indizes with different shard size?