I am missing something very basic when I think of how Filebeat will be configured in a clustered logstash setup.
As per the article https://www.elastic.co/guide/en/logstash/current/deploying-and-scaling.html and this architecture diagram in the article, I think that there is some kind of load balancer in front of the logstash cluster. However, the Filebeat output documentation suggests that there must be an array of all the Logstatsh nodes specified. Using this list of nodes, Filebeat will do the load balancing from the client-side.
Also as per https://github.com/elastic/logstash/issues/2632 GitHub issue, there is no native logstash clustering available yet.
So, my question is, what kind of setup do I need to be able to point my multiple Filebeat to one logstash service endpoint without specifying the logstash nodes in the cluster?
- Is it possible?
- Would having load balancer in front of Logstash cluster be of any help?