Syslog input doesn't seem to work, rsyslog error -2027

Elastic Stack Beats
1

Hey all,

I'm having issues getting the Filebeat syslog input plugin to receive remote syslog messages.

My filebeat config looks like:

filebeat:
  config:
    modules:
      path: ${path.config}/modules.d/*.yml
      reload:
        enabled: false
  inputs:
    - type: syslog
      protocol.tcp:
        host: ":5514"
      tags: ["remote-syslog","remote-syslog-tcp"]
    - type: syslog
      protocol.udp:
        host: ":5514"
      tags: ["remote-syslog","remote-syslog-udp"]

When I configure rsyslog on a vagrant vm to forward logs to my host's internal IP, I get errors like this:

Sep  7 17:25:01 bionic rsyslogd: omfwd: TCPSendBuf error -2027, destruct TCP Connection to 192.168.1.100:5514 [v8.32.0 try http://www.rsyslog.com/e/2027 ]
Sep  7 17:25:01 bionic rsyslogd: action 'action 7' suspended (module 'builtin:omfwd'), retry 0. There should be messages before this one giving the reason for suspension. [v8.32.0 try http://www.rsyslog.com/e/2007 ]
Sep  7 17:25:01 bionic rsyslogd: action 'action 7' resumed (module 'builtin:omfwd') [v8.32.0 try http://www.rsyslog.com/e/2359 ]

Though, I only sometimes see those logs. Other times I just tail syslog, see messages being written, but nothing shows up in Kibana's Discover section when looking at the filebeat-* index.

From what I've found via searching, I think those errors mean there must be something interfering with the connection between rsyslog and Filebeat.

But what?

It shouldn't be the firewall. I installed nmap on my vm. It reports 5514 is open.

If I nmap 5514 on my host, it reports as open.

That said, I am running firewalld, so I added 5514/tcp and 5514/udp to the list of allowed public zone ports.

I did try turning off the firewall, but it made no difference.

Are there any other firewall settings that might be getting in the way?

I do know that Filebeat can receive remote syslog messages. If I configure my host's rsyslog to forward to Filebeat on 5514, it works. Both using 127.0.0.1 and my hosts LAN ip. It's the LAN ip I've configured my vm to use.

I'd blame something with my vm setup, but I've configured my router to forward logs as well. I don't see anything from it either. (Unless dd-wrt can't send to a custom port?)

At this point I'm really not sure what else to try. Any suggestions?

2

Is that valid? I haven't seen it before.

3

It doesn't throw an error.

Is there another way to tell Filebeat to listen on all interfaces? I think I tried "0.0.0.0:5514" and Filebeat wouldn't start.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.