Loadbalance Logstash without double ES-data?

Elastic Stack Logstash
1

Hi,
The architecture is not done yet, but i want to ask so i know how to Think.

Lets say i have:
10 clients i want to collect logs from, which will have beatagents.
2 Logstash servers that the beatagents will send to
3 Elasticsearch nodes in a cluster.

If i configure the two logstash servers to send incoming data to 2 of the elasticsearch servers.
Will i get double events in ElasticSearch?

How should i Think here.. i want redundant logstash serversn as i do with ElasticSearch.

2

If i configure the two logstash servers to send incoming data to 2 of the elasticsearch servers.
Will i get double events in Elasticsearch?

No, because the beats agent won't sent the same events to both Logstash instances.

3

ok, so if i configure two logstash servers for a beat agents they will try to send to the first, and if they succeed they move on to the next document. But if the first logstash failes to recieve, they move on after x amount of tries to the second logstash server?

4

Found the answer to my own question :slight_smile:

loadbalanceedit
If set to true and multiple Logstash hosts are configured, the output plugin load balances published events onto all Logstash hosts. If set to false, the output plugin sends all events to only one host (determined at random) and will switch to another host if the selected one becomes unresponsive. The default value is false.

Really impressed by elastics documentation.:heart_eyes:

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.