Logstash is not accepting logs from rsyslog

Elastic Stack Logstash
1

Environment:

  • 50 CentOS 7.6 servers
  • 192.168.1.145 - Elastic Search master server
  • 192.168.1.146 - Elastic Search master server
  • 192.168.1.147 - Elastic Search master server
  • 192.168.1.148 - Elastic Search data server
  • 192.168.1.149 - Elastic Search data server
  • 192.168.1.150 - Logstash and rsyslog server
  • 192.168.1.151 - Kibana server

I was following this blog posting:

I followed all the steps and all my servers are logging to the rsyslog server so that's working.

Per the blog, I have:

# This input block will listen on port 10514 for logs to come in.
# host should be an IP on the Logstash server.
# codec => "json" indicates that we expect the lines we're receiving to be in JSON format
# type => "rsyslog" is an optional identifier to help identify messaging streams in the pipeline.
input {
  udp {
    host => "192.168.1.150"
    port => 10514
    codec => "json"
    type => "rsyslog"
  }
}
# This is an empty filter block.  You can later add other filters here to further process
# your log lines
filter { }
# This output block will send all events of type "rsyslog" to Elasticsearch at the configured
# host and port into daily indices of the pattern, "rsyslog-YYYY.MM.DD"
output {
  if [type] == "rsyslog" {
    elasticsearch {
      hosts => [ "192.168.1.145:9200", "192.168.1.146:9200", "192.168.1.147:9200" ]
    }
  }
}

and

# This line sends all lines to defined IP address at port 10514,
# using the "json-template" format template 
*.*                         @192.168.1.150:10514;json-template

Logs are showing logstash connecting to the three Elastic Search servers.

I've done some troubleshooting, changing the host ip in the configuration files to 127.0.0.1 per another comment I read, put the port into quotes just in case, changed rsyslog to syslog in case there's a type error, but it all looks like it's configured correctly.

2

Hi,

if You have SELinux set to "enforcing" mode on the rsyslog server, You will need to allow the rsyslog process to send to destination port 10514:

semanage -a -t syslogd_port_t -p udp 10514

Personally, I would send the logs via TCP from rsyslogd to Logstash with the tcp input and the rsyslog forwarding configured:

*.*           @@192.168.1.150:10514;json-template

(the semanage command would need to be altered accordingly)

But that's personal preference, although some firewalls don't really like UDP packets.

3

Hi,

SELinux is disabled on the server in question (and all the others).

I think I tried tcp (the @@) on both configurations and then back to udp but neither worked. I'll verify it when I get home though just to be sure.

Note that the rsyslog configuration and logstash are both on the same server and that rsyslog is receiving logs from all 50 servers.

Thanks for replying.

Carl

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.