Not getting a listening port for Beats Input with Logstash

Elastic Stack Logstash
1

Hello,

I have logstash installed in my enviornment and it currently only has 1 pipeline setup for syslog messages, this one pipeline is sucessfully opening a listening port for the logs to be received.

I setup a second pipeline for packetbeats to forward logs to logstash.

The working syslog server pipeline is shown here:
#Unencrypted syslog
input {
tcp {
type => "syslog"
port => 6514
}
}
filter {
if [type] == "syslog" {
grok {
match => {
"message" => "(?m)<%{POSINT:syslog_pri}>(?:%{SYSLOGTIMESTAMP:syslog_timestamp}|%{TIMESTAMP_ISO8601:syslog_timestamp}) %{SYSLOGHOST:syslog_hostname} %{SYSLOGPROG}%{DATA}[\s\n]%{GREEDYDATA:syslog_message}" }
}
mutate { add_tag => "linux-syslog" }

    if ("_grokparsefailure" in [tags]) {
    #solaris
        grok {
            match => {
                "message" => ["(?m)<%{POSINT:syslog_pri}>(?:%{SYSLOGTIMESTAMP:syslog_timestamp}|%{TIMESTAMP_ISO8601:syslog_timestamp}) %{SYSLOGPROG}%{DATA}[\s\n]%{GREEDYDATA:syslog_message}"]}
            add_field => { "source_type" => "solaris" }
            remove_tag => [ "_grokparsefailure" ]
        }
    }
    grok { match => { "host" => "(?:%{IPV4:received_from_ipv4}|%{IPV6:received_from_ipv6})(:%{POSINT})?" } }
    syslog_pri { }
    date { match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss", "ISO8601" ] }
    mutate { remove_field => ["syslog_timestamp", "syslog_pri", "host"] }
}

}
output {
file {
path => "/data/logs/all.log"
}
elasticsearch {
index => "logstash-%{+YYYY.MM.dd}"
hosts => ["https://x.x.x.x:9200", "https://x.x.x.x:9200", "https://x.x.x.x:9200"]
ssl => true
cacert => '/etc/logstash/certs/ca/ca.crt'
user => 'elastic'
password => 'PASSWORD'
}
}

This is the packetbeat pipeline that does not open a logstash port.

input {
beats {
port => 5554
}
}

output {

    elasticsearch {
            index => "logstash-%{+YYYY.MM.dd}"
            hosts => ["https://x.x.x.x:9200", "https://x.x.x.x:9200", "https://x.x.x.x:9200"]
            ssl => true
            user => 'elastic'
            password => 'PASSWORD'
    }

}

---so basically logstash starts up fine, opens the syslog pipeline port, never opens the packetbeat port.

Any thoughts?

2

From only a configuration it is hard to tell why this should not work. From what I can see you config looks ok.

Are there no errors / warnings in the logstash log at al? It seems weird that logstash loads fine and not open up the requested port.

Paul.

3

The logs shown in logstash-plain.log do not show any errors or warning when starting, but it also does not seem to display any information in regards to the new pipeline. Nothing states whether it had any issues or even attempted to start..very weird.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.