Hey, Elastic newbie here.
I have few indices which will store LOTS of data (50GB + ) compared to other indices which store much less (1-2GB tops).
All the indices share the same default configuration (5 shards, 1 replica) and I think it won't be optimal in the future.
- My actual question is - How should I divide my resources and decide how many shards and replicas every index has?
I'm currently running an elastic cluster on 4 nodes (servers) and am using the default settings in basically everything.
I've had no problems so far, but I'm trying to look ahead and I've come to realize that my indices are probably configured incorrectly.
Each of my clusters has 32GB RAM, 16 cores processors and around 4TB HD.
I read a few articles online that recommend to overallocate shards in indices which are expected to hold lots of data (and when your hardware can afford it, and I think my hardware can), but I'm still not sure on how to determine these settings.
so again, my question is - How should I divide my resources and decide how many shards and replicas every index has?
Thanks in advance,