The cluster has 5 hot nodes, 3 warm nodes and 3 dedicated master nodes (on-premise). We process the logs from multiple applications using Beats plugin. As we have option in Beats to add array of data nodes with loadbalance turned on.
Based on the below criteria what's the best practice recommendation for load balancer.
Failover and recover.
No data loss even when a node goes down.
How to avoid extra overhead when we add more nodes to the cluster for scaling. Because with array of hosts we need to add new node when we include new node to the cluster and restart beats in every application.
Please suggest either to use array of hosts or separate load balancer ?
@val - Appreciate, if you could provide your inputs.
What we usually do in this case is to hide all the ES nodes behind a load-balancer and reference the load balancer inside the Beats configuration. That gives you the flexibility to grow/shrink your cluster as you see fit without having to restart all your beats fleet.
From the official doc
If one node becomes unreachable, the event is automatically sent to another node.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.