How to handle multiple syslog inputs with Logstash to different indices

(Влад Радин) #1

Hi, in input I use 2 ports by 2 project, I want handle them to different indices. But now all input go to prd.rnis-* index. Can you help?

input {
        syslog {
                port => 5001
                tags => ["prd.rnis"]
                grok_pattern => "<%{NONNEGINT:syslog_pri}>%{NONNEGINT:version}%{SPACE}(?:-|%{TIMESTAMP_ISO8601:syslog_timestamp})%{SPACE}(?:-|%{IPORHOST:hostname})%{SPACE}(?:%{SYSLOG5424PRINTASCII:program}|-)%{SPACE}(?:-|%{SYSLOG5424PRINTASCII:process_id})%{SPACE}(?:-|%{SYSLOG5424PRINTASCII:message_id})%{SPACE}(?:-|(?<structured_data>(\[.*?[^\\]\])+))(?:%{SPACE}%{GREEDYDATA:syslog_message}|)"
        }
        syslog {
                port => 5002
                tags => ["dem.nisr"]
                grok_pattern => "<%{NONNEGINT:syslog_pri}>%{NONNEGINT:version}%{SPACE}(?:-|%{TIMESTAMP_ISO8601:syslog_timestamp})%{SPACE}(?:-|%{IPORHOST:hostname})%{SPACE}(?:%{SYSLOG5424PRINTASCII:program}|-)%{SPACE}(?:-|%{SYSLOG5424PRINTASCII:process_id})%{SPACE}(?:-|%{SYSLOG5424PRINTASCII:message_id})%{SPACE}(?:-|(?<structured_data>(\[.*?[^\\]\])+))(?:%{SPACE}%{GREEDYDATA:syslog_message}|)"
        }
}
filter {

                json {
                   source => "syslog_message"
                     }
                }


output {
        if "prd.rnis" in [tags] {
            elasticsearch {
                hosts => [ "elasticsearch:9200" ]
                index => "prd.rnis-%{+YYYY.MM.dd}"
                }}
        if "dem.nisr" in [tags] {
            elasticsearch {
                hosts => [ "elasticsearch:9200" ]
                index => "dem.nisr-%{+YYYY.MM.dd}"
                }}

  stdout { codec => rubydebug }

}
#2

Are you saying that there are documents in the prd.rnis index that do not have a prd.rnis tag?

(Влад Радин) #3

No, I say, that I don't have documents in index dem.nisr. Documents in prd.rnis index only.

#4

That suggests nothing is arriving on port 5002.

(Влад Радин) #5

Hmm.. But I see traffic in tcpdump..
Ok, I wanted be sure, that my config is good.
I try fix it.