I have a cluster consisting of 4 data and 1 master node
each node has:
cpu: 4
ram: 16
and we have 1 largest index of 215 gigabytes (1 primary and 1 replica shard)
on peak days, this index is very heavily loaded (we use aggregation queries, since the index is used to store and send notifications to users), which negatively affects the operation of the entire cluster
the developers propose to allocate a separate cluster for this index, and I propose to add 3 more machines to the cluster and use shard allocation awareness only for this index and divide it into 3 primary and 1 replica shard
what do you think this is right approach? or what is the best way to do it in your opinion?
First I suggest you read this very informative section on sizing shards here
Just a quick analysis I would break that index up into probably at least 5 shards with one replica, to get those shards down below 50 gigabytes per shard.
Also does That index need to be in a single index with an index pattern perhaps you could break that index into multiple indexes And as well as reducing the shard size.
After the allocation awareness there's some pros and cons to that now you'll be limiting down to less nodes which means less primary shards and may not suit your purpose.
Second if this is a production cluster and you only have one master node you are it at risk of cluster failure if you lose that masternode.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.