Hey, guys, we have some concerns about how many client nodes should be in a es cluster.
Now we have a es cluster with total 36 nodes: 3 master nodes, 5 client nodes and 28 datanodes. We have approximate 10TB, which about 14 billions data in our cluster, the indexing rate is about 20k in average and can reach 40k in peek, the search rate is about 10k and can reach 20k in peek.
Recently we receive some error about circuit_breaking now and then, after some digging we found that this is because the jvm heap usage of client node is high and finally trigger circuit_breaking. Then we add 2 more client nodes into the cluster and set InitiatingHeapOccupancyPercent=30 and set G1ReservePercent=25, after there operations we finally solve the circuit_breaking problem.
Though the problem is issued, we still have concern about how many client nodes should be in a es cluster. I know that the number of master node should be 3 no matter the cluster size, and number of data node should vary with the data size and pressure of whole cluster, but I cannot find any information about how many client nodes should be in a cluster. Is there any suggestion about this? Or do you do some pressure test about how many client nodes should be in a real es cluster? Thank you very much!!
Hi,
It is a combination of how many shard and indices you have. Can you share some information about that?
Paul.
We have about 800 indices total and about 2500 primary shards, each primary shard have a replica shard
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.