Previously, we have been running 6 different logstash processes on each Logstash server (5 shippers and 1 indexer) and 1 Redis process. The shippers each had their own configuration file, listening for data incoming on specific ports, and the output section was configured to send to one of two Redis servers (for load balancing and failover protection). The indexer process then had an input of Redis and output to Elasticsearch.
With ES 2.x, we decided to standardize and use a single logstash.conf file and therefore a single logstash process + 1 Redis process. Which should make things easier to manage, handle, and monitor. But I can't find any examples of using Redis in the middle like this.
But how do I specify multiple options for the output section. Specifically, what values can I key off of to specify that the shippers (syslog, snmptraps, beats, logs, etc) all ship their data to Redis first. Then only redis output gets sent to Elasticsearch.