Clustering Logstash shippers

Elastic Stack Logstash
1

Hi,
We are preparing our environment to Logstash deployment and i have one question about clustering of Logstash shippers:
On all the documents i have read about clustering Logstash i saw that we should have 2 or more shippers, but i don't understand how can i send each message to only one shipper? the only option i saw is to send each message to both of the shippers servers and use fingerprint filter as the Document ID of the message when sending it to elasticsearch and then the message will be overwritten..
there is any other option so i'll send each message just one time?

Thanks!

2

someone?

3

Usually, a "shipper" is what runs on the leaf machines where the logs are collected. They send their logs to a smaller number of instances, frequently referred to as "indexers", which I think is a really terrible name (I prefer "hub").

Anyway.

On all the documents i have read about clustering Logstash i saw that we should have 2 or more shippers, but i don't understand how can i send each message to only one shipper?

That depends on how you're sending the events. Some output allow you to list multiple hosts for a given output plugin, in which case the plugin will use either or. Another frequently used option is to use a message broker as a buffer for the incoming events and have the receiving Logstash instances pull from the same queue (with the broker guaranteeing that a message isn't delivered twice).