I'm setting up an ELK which will take a feed of syslogs from a variety of devices including Cisco's Linux Server Syslogs, App files and Windows server logs and files.
I want to make it as redundant as possible so that we can send syslogs from the network devices (and servers etc) to two aggregation points but avoid duplication of lines.
So for syslogs I am defining the devices to send to two syslog-ng nodes. The part I need to work out is how I can then get both syslog-ng nodes to send to logstash (2 nodes) but not duplicate the log entries.
Is there a mechanism I can use so I can have a beat(?) on each syslog-ng sending to logstash (using the redundancy option) but avoid getting duplicated syslogs in the ES at the final aggregation.