Configuring logstash-input-syslog

Elastic Stack Logstash
1

Hi guys! I absolutely love the ELK stack - please keep up the great work.

Background: We have an Ubuntu 16.04 rsyslog server collecting syslogs from various servers and network equipment. I've been tasked with trying to get ELK to present those logs (as well as Windows Events and application logs eventually). I installed Elasticsearch, Kibana, Logstash, and Filebeat on the syslog server.

At first I configured filebeat to read /var/log/syslog which contained all the logs received from any host. My /etc/filebeat/filebeat.yml had elasticsearch AND logstash as outputs, which I still don't understand (wouldn't you just want logstash as the destination for filebeat, and logstash in turn will send to elasticsearch which will be queried by kibana?? setup.kibana was also configured, further adding to my confusion. I'll ask for some clarification about all that in the appropriate sections.)

In any case that was working, but in kibana the beat.hostname was always that of the syslog/ELK server. The log sender is mentioned in the message itself, but that's not an indexed field and isn't very useful for when sifting through the info.

I finally came across the logstash-input-syslog plugin today and realized this should solve that issue - logstash will hopefully parse which host sent each message it receives. I stopped listening on 514/udp with rsyslog and added the following syslog-input.conf file to /etc/logstash/conf.d/

input {
syslog {
port => 514
type => “syslog_server”
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
}
}

... but now "netstat -npl" doesn't show it listening on 514 tcp or udp. "ps -ef" shows logstash running as the "logstash" user, so do I just need to set a sticky bit or something in order to allow that user to bind to port 514?

Thanks in advance for any help you can provide!
-static-

2

P.S. I realize I could try a port greater than 1024 but no one here wants to have to reconfigure the 100+ routers and VPN devices nor the 60 or so servers; therefore, it would be easier to just slip logstash in as a replacement for rsyslog.

3

Only root can bind to ports < 1024. The easiest way out is probably to use iptables to redirect port 514 to something that Logstash can bind to.

4

OK thanks for the reply! I stumbled on the following after posting, once I tried a slightly more intelligent google search. In case it helps anyone else... It's regarding the same issue I've asked about with a bit more info.

Unfortunately my idea to set a sticky bit would NOT be advisable for this because judging by my "ps -ef" output it's actually java that binds to the port, not logstash directly; meaning if one were to set a sticky bit for java, ANY java applet/servlet could bind to any port it wanted. FYI

5

Use iptables to redirect traffic arriving on 514 to another port, e.g. 55514, and have logstash listen on this other port.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.