Horizontally scaling Logstash - Per User Guide

I have been working with the ELK stack for quite some time, but now that I am going to production I need to scale Logstash for larger throughput. I have seen the guide which shows the distributed layout:


However, I have never seen example configs/code that shows how this works. I need to listen on 514 for syslog but a single logstash instance cannot handle the volume my system is producing. Also note that the customer will not allow install of any collector on their servers (i.e. no Beats) :frowning: We have to be able to listen on a single port as if we were a logging server.

If I set each of my logstash instances to listen of port 514, how do I prevent message duplication. The diagram in the guide suggests that you can send data to many instances and then into a queue.

Thanks guys.


You'd want a load balancer that listens on the port and feeds to multiple LS instances. Something like haproxy would work.