Loadbalancing with syslog

[trenzalore]

Hi !

I use a syslog server which sends log to my cluster (3 servers cluster). Each server has a logstash instance.
The syslog server send only to 2 servers.

I found that my entries are doubled and I think it's because the syslog server send the same logs to the two servers.

So I would like to know if there is a way to send to only one IP which will send to the three logstash servers. I thought about redis but I've nerver used it and don't know how it works.

Thanks for your help !

[eperry (Ed)]

You will have to set up some kind of Load Balancer, if Syslog does not support something like this

Fortunately your dealing with syslog which can do TCP or UDP Connections

You could try a few options, I use Hardware like an F5 but any hardware would work

But you can do it in software too
http://haproxy.com/ (TCP or UDP)
Apache (TCP only I think)
Nginx https://www.nginx.com/resources/admin-guide/tcp-load-balancing/ (or udp)

Probably the easiest but most technical

Iptables
http://lartc.org/autoloadbalance.htmlroxy

Other then that a little googling will go a long long way.

[trenzalore]

In fact I'm trying something totally different.

I only put one logstash instance, every logs will be send to it and the output of the config files will do the loadbalancing (I wrote de name of the different elasticsearch hosts)

I think it's a good solution to my problem.

[eperry (Ed)]

Great