I have two hot nodes in my cluster. I currently just have on node in the Outputs. Can I just add the 2nd node to the list of outputs?
Some clarification, I have one URL specified in the Fleet configuration outputs. I would like to add the URL for the 2nd node so the data going to the cluster is load balanced.
AFAK fleet will attempt to use the first endpoint in the list and only if that fails use the next.
Cleaner option (if you don't have coordination nodes who take care of this for you) is to use a loadbalancer which round-robins to your hot nodes. You then configure the loadbalancer as output.
Just edit the output and add the second hot node then fleet will load balance the requests per default.
This is only true for the Fleet Server communication, for the data output, e.g. integration data, it has the same behavior as beats and will load balance between all the nodes listed.
2 Likes
According to the docs, this is not the default behavior: Logstash output | Fleet and Elastic Agent Guide [8.11] | Elastic
host: (list) The list of known Logstash servers to connect to. If load balancing is disabled, but multiple hosts are configured, one host is selected randomly (there is no precedence). If one host becomes unreachable, another one is selected randomly.
Load balancing is disabled by default.
This is just for the Logstash output, not the Elasticsearch output.
While using the Elasticsearch output load balance is enabled by default.
The list of Elasticsearch nodes to connect to. The events are distributed to these nodes in round robin order.
It is the same behaviour as beats, Elasticsearch load balances per default, Logstash needs to configure the loadbalance, pipelining and ttl settings.
ah ok, understood.
Thanks!!
Thanks for the input guys. I've added the 2nd Output in Fleet configuration.
However; I'm not able to select the 2nd Output in an integration policy. the new option is listed, but the GUI doesn't allow me to select it.
When I try enabling the default options in the new output, I get an error telling me the fleet-default config can only be updated in the Kibana config file.
I'm going to try adding my 2nd Output to the Kibana config file and see if that gives me better results.
Are you using Fleet manage agents or standalone manages?
With Fleet Manage everything is configured through the Fleet UI.
Can you share a screenshot of your error?
I've also tried adding a 2nd default to the kibana.yml but I get an error when I restart the kibana service.
FATAL Error: [config validation of [xpack.fleet].outputs]: preconfigured outputs can only have one default output.
So you are not using Fleet managed agents.
I think you need to configure the output like the example in the documentation, but your second hot node in the array for the hosts setting.
I do not use Elastic Agent/Fleet as standalone, so can not help further with this configuration.
According to the documentation, I added the 2nd IP address to the list of hosts.
That worked, and I confirmed connections from agents to both nodes.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.