Horizontally scaling Logstash - Per User Guide

Elastic Stack Logstash
1

I have been working with the ELK stack for quite some time, but now that I am going to production I need to scale Logstash for larger throughput. I have seen the guide which shows the distributed layout:

https://www.elastic.co/guide/en/logstash/current/deploying-and-scaling.html#deploying-logstash-ha

However, I have never seen example configs/code that shows how this works. I need to listen on 514 for syslog but a single logstash instance cannot handle the volume my system is producing. Also note that the customer will not allow install of any collector on their servers (i.e. no Beats) :frowning: We have to be able to listen on a single port as if we were a logging server.

If I set each of my logstash instances to listen of port 514, how do I prevent message duplication. The diagram in the guide suggests that you can send data to many instances and then into a queue.

Thanks guys.

-Jonathan

2

You'd want a load balancer that listens on the port and feeds to multiple LS instances. Something like haproxy would work.