# Load Balance Output to both hot nodes

**URL:** https://discuss.elastic.co/t/load-balance-output-to-both-hot-nodes/350415
**Category:** Elastic Agent
**Created:** [January 4, 2024, 9:25pm UTC](https://discuss.elastic.co/t/load-balance-output-to-both-hot-nodes/350415 "2024-01-04T21:25:17Z")
**Posts on this page:** 15
**Page:** 1

<div class="post-metadata">

### Author: ![sourcreamnormanbates](https://avatars.discourse-cdn.com/v4/letter/s/f05b48/32.png) [@sourcreamnormanbates](https://discuss.elastic.co/u/sourcreamnormanbates)
#### Post date: [January 4, 2024, 9:25pm UTC](https://discuss.elastic.co/t/load-balance-output-to-both-hot-nodes/350415/1 "2024-01-04T21:25:17Z")

</div>

I have two hot nodes in my cluster. I currently just have on node in the Outputs. Can I just add the 2nd node to the list of outputs?

---

<div class="post-metadata">

### Author: ![sourcreamnormanbates](https://avatars.discourse-cdn.com/v4/letter/s/f05b48/32.png) [@sourcreamnormanbates](https://discuss.elastic.co/u/sourcreamnormanbates)
#### Post date: [January 5, 2024, 2:08pm UTC](https://discuss.elastic.co/t/load-balance-output-to-both-hot-nodes/350415/2 "2024-01-05T14:08:05Z")

</div>

Some clarification, I have one URL specified in the Fleet configuration outputs. I would like to add the URL for the 2nd node so the data going to the cluster is load balanced.

---

<div class="post-metadata">

### Author: ![sholzhauer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sholzhauer/32/110282_2.png) [@sholzhauer](https://discuss.elastic.co/u/sholzhauer)
#### Post date: [January 5, 2024, 6:28pm UTC](https://discuss.elastic.co/t/load-balance-output-to-both-hot-nodes/350415/3 "2024-01-05T18:28:03Z")

</div>

AFAK fleet will attempt to use the first endpoint in the list and only if that fails use the next.

Cleaner option (if you don't have coordination nodes who take care of this for you) is to use a loadbalancer which round-robins to your hot nodes. You then configure the loadbalancer as output.

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [January 6, 2024, 12:36pm UTC](https://discuss.elastic.co/t/load-balance-output-to-both-hot-nodes/350415/4 "2024-01-06T12:36:23Z")

</div>

> [@sourcreamnormanbates](#):
>
> I have one URL specified in the Fleet configuration outputs. I would like to add the URL for the 2nd node so the data going to the cluster is load balanced.

Just edit the output and add the second hot node then fleet will load balance the requests per default.

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [January 6, 2024, 12:38pm UTC](https://discuss.elastic.co/t/load-balance-output-to-both-hot-nodes/350415/5 "2024-01-06T12:38:20Z")

</div>

> [@sholzhauer](#):
>
> AFAK fleet will attempt to use the first endpoint in the list and only if that fails use the next.

This is only true for the Fleet Server communication, for the data output, e.g. integration data, it has the same behavior as beats and will load balance between all the nodes listed.

---

<div class="post-metadata">

### Author: ![DefensiveDepth](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/defensivedepth/32/110034_2.png) [@DefensiveDepth](https://discuss.elastic.co/u/DefensiveDepth)
#### Post date: [January 8, 2024, 1:54pm UTC](https://discuss.elastic.co/t/load-balance-output-to-both-hot-nodes/350415/6 "2024-01-08T13:54:18Z")

</div>

According to the docs, this is not the default behavior: [Logstash output | Fleet and Elastic Agent Guide [8.11] | Elastic](https://www.elastic.co/guide/en/fleet/current/logstash-output.html#output-logstash-commonly-used-settings)

` host: (list) The list of known Logstash servers to connect to. If load balancing is disabled, but multiple hosts are configured, one host is selected randomly (there is no precedence). If one host becomes unreachable, another one is selected randomly.`

Load balancing is disabled by default.

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [January 8, 2024, 2:04pm UTC](https://discuss.elastic.co/t/load-balance-output-to-both-hot-nodes/350415/7 "2024-01-08T14:04:38Z")

</div>

> [@DefensiveDepth](#):
>
> Load balancing is disabled by default.

This is just for the Logstash output, not the Elasticsearch output.

While using the Elasticsearch output load balance is [enabled by default](https://www.elastic.co/guide/en/fleet/current/elasticsearch-output.html#output-elasticsearch-commonly-used-settings).

> The list of Elasticsearch nodes to connect to. The events are distributed to these nodes in round robin order.

It is the same behaviour as beats, Elasticsearch load balances per default, Logstash needs to configure the `loadbalance`, `pipelining` and `ttl` settings.

---

<div class="post-metadata">

### Author: ![DefensiveDepth](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/defensivedepth/32/110034_2.png) [@DefensiveDepth](https://discuss.elastic.co/u/DefensiveDepth)
#### Post date: [January 8, 2024, 2:07pm UTC](https://discuss.elastic.co/t/load-balance-output-to-both-hot-nodes/350415/8 "2024-01-08T14:07:13Z")

</div>

ah ok, understood.

Thanks!!

---

<div class="post-metadata">

### Author: ![sourcreamnormanbates](https://avatars.discourse-cdn.com/v4/letter/s/f05b48/32.png) [@sourcreamnormanbates](https://discuss.elastic.co/u/sourcreamnormanbates)
#### Post date: [January 9, 2024, 1:34pm UTC](https://discuss.elastic.co/t/load-balance-output-to-both-hot-nodes/350415/9 "2024-01-09T13:34:36Z")

</div>

Thanks for the input guys. I've added the 2nd Output in Fleet configuration.  
However; I'm not able to select the 2nd Output in an integration policy. the new option is listed, but the GUI doesn't allow me to select it.

When I try enabling the default options in the new output, I get an error telling me the fleet-default config can only be updated in the Kibana config file.

I'm going to try adding my 2nd Output to the Kibana config file and see if that gives me better results.

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [January 9, 2024, 1:40pm UTC](https://discuss.elastic.co/t/load-balance-output-to-both-hot-nodes/350415/10 "2024-01-09T13:40:32Z")

</div>

Are you using Fleet manage agents or standalone manages?

With Fleet Manage everything is configured through the Fleet UI.

Can you share a screenshot of your error?

---

<div class="post-metadata">

### Author: ![sourcreamnormanbates](https://avatars.discourse-cdn.com/v4/letter/s/f05b48/32.png) [@sourcreamnormanbates](https://discuss.elastic.co/u/sourcreamnormanbates)
#### Post date: [January 9, 2024, 2:04pm UTC](https://discuss.elastic.co/t/load-balance-output-to-both-hot-nodes/350415/11 "2024-01-09T14:04:58Z")

</div>

![image](https://us1.discourse-cdn.com/elastic/original/3X/2/a/2ab66bd55fcd6f998bbbb03724a55c5ebcc258ab.png)

---

<div class="post-metadata">

### Author: ![sourcreamnormanbates](https://avatars.discourse-cdn.com/v4/letter/s/f05b48/32.png) [@sourcreamnormanbates](https://discuss.elastic.co/u/sourcreamnormanbates)
#### Post date: [January 9, 2024, 3:54pm UTC](https://discuss.elastic.co/t/load-balance-output-to-both-hot-nodes/350415/12 "2024-01-09T15:54:27Z")

</div>

I've also tried adding a 2nd default to the kibana.yml but I get an error when I restart the kibana service.

FATAL Error: [config validation of [xpack.fleet].outputs]: preconfigured outputs can only have one default output.

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [January 9, 2024, 4:17pm UTC](https://discuss.elastic.co/t/load-balance-output-to-both-hot-nodes/350415/13 "2024-01-09T16:17:05Z")

</div>

So you are not using Fleet managed agents.

I think you need to configure the output like the example in the [documentation](https://www.elastic.co/guide/en/fleet/current/elasticsearch-output.html#output-elasticsearch-commonly-used-settings), but your second hot node in the array for the `hosts` setting.

I do not use Elastic Agent/Fleet as standalone, so can not help further with this configuration.

---

<div class="post-metadata">

### Author: ![sourcreamnormanbates](https://avatars.discourse-cdn.com/v4/letter/s/f05b48/32.png) [@sourcreamnormanbates](https://discuss.elastic.co/u/sourcreamnormanbates)
#### Post date: [January 9, 2024, 5:53pm UTC](https://discuss.elastic.co/t/load-balance-output-to-both-hot-nodes/350415/14 "2024-01-09T17:53:30Z")

</div>

According to the documentation, I added the 2nd IP address to the list of hosts.

That worked, and I confirmed connections from agents to both nodes.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 6, 2024, 5:53pm UTC](https://discuss.elastic.co/t/load-balance-output-to-both-hot-nodes/350415/15 "2024-02-06T17:53:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
