Logstash sending to all output hosts

Elastic Stack Logstash
1

So, here is a really odd situation. I have two pipeline files:

  1. Receive "Beats" fwd'ed logs over 5044 - send to ElasticSearch (x.x.x.164)
 input {
    beats {
        port => "5044"
    }
}
output {
  elasticsearch {
    hosts => ["x.x.x.164:9200"]
    user => "xxxx"
    password => "xxxx"
  }
  #stdout { codec => rubydebug }
}
  1. Receive Splunk "uncooked" log fwds over 2026, send to Splunk server (x.x.x.153) over UDP port 5014
input {
        udp{
                port => 2026
        }

        tcp{
                port => 2026
        }
}



output {
                udp {

                                        host => ["x.x.x.153"]
                                        port => "5014"

                        }
                }

Soooo....Both Elastic and Splunk are getting the same logs....if I delete the "Beats" pipeline and restart Logstash, Splunk just gets the "uncooked" log data coming in over 2026 and Elastic gets nothing (makes sense).

What is a mystery is why is Logstash is ignoring the "Beats" Pipeline directive (only logs coming in over 5044) and sending "Beats" logs over to Splunk as well?

2

Unless specifically configured otherwise Logstash has a single event pipeline. All files in the conf.d directory are concatenated. All events from all inputs get passed to all filters and all outputs. If that's not desirable you need to add conditionals to route events the way you want.

3

Ah...that makes a ton of sense....

Not asking for a solution (as I need to learn this stuff), but a nod in the right direction would be appreciated greatly.

4

I put the following directive in the Splunk pipeline conf file and it seemed to do the trick...


filter{
    if "beats_input_codec_plain_applied" in [tags] { drop {}}

Will ping back if it also stops the logs going to Elastic....

5

So, here is what finally worked for anyone that runs across this...


input {
        udp{
                port => 2026
        }

        tcp{
                port => 2026
        }
}



output {

if "beats_input_codec_plain_applied" in [tags] {

elasticsearch {
    hosts => ["x.x.x.164:9200"]
    user => "elastic"
    password => "changeme"
  }



} else {
                udp {

                                        host => ["x.x.x.153"]
                                        port => "5014"

                        }
                }
        }

Only "Beats" logs are going to ES and the HF stuff is the only thing showing up in Splunk.....