Load-Balancing the Elastic-Service

[Moritz_Kiesewetter]

Hi Guys,
so i've recently picked up ElasticStack at my new WorkPlace, and now i have to optimize the Cluster which was built by a previous co-worker. The Cluster consits of a Master Node (6.8v) with the Kibana Installation, an Appache WebServer (Which just now also does the AUTH).

The Worker Nodes (#01 , #02 ) both have the Logstash & Elasticsearch packets installed and configured. Also they're running with a keepalived deamon to grant HA.

Now my Problem ist that Node #01 is currently doing all the Work (around 86-92% CPU-Usage.), whilst #02 is just chillin around.

Is there already an established way for load balancing in an ElasticStack cluster?
If not, which method did you use? Or with which method did you have the best experience?

Thanks for the help as always!
If i posted this in the wrong category, sorry in advance!

Greetz
Mo

[Antoine.E (Antoine Espinosa)]

Hello Moritz,

From my experience most of the CPU is probably used by logstash if you use the grok filter, I would advise you to put a load balancer in round robin in front of your two nodes so that the load of the parsing is balanced.

If Logstash is not the service taking most of the CPU, I am not sure if this setup would work so look on your node #1 what is the most CPU intensive service.

Good luck,
Antoine