I am trying to setup a rsyslog+ELK stack following this tutorial without success:
My configuration is similar to the one in the tutorial except that the rsyslog server and the whole ELK stack are running on the same server.
Note that the ELK stack was previously used with Filebeat and it was working fine.
The rsyslog server receive the logs from the client and generate the template as expected. It seems to also send the logs to logstash properly according to rsyslog debug logs:
"ACTION 0x1bc5a40 [builtin:omfwd:@localhost:10514;json-template]"
Full rsyslog server logs generated when "sudo ls" is executed on the rsyslog client are available here: https://www.zerobin.net/?1cc8d26e568ae70f#EAnAGTD+pZUMgh0U8Ff7E2jCWU3/vSdZOs4KxXP33aU=
However, I can't find the logs in elasticsearch and I don't see anything happening in logstash logs, even though it is listening to the correct port:
$ sudo netstat -na | grep 10514
udp6 0 0 127.0.0.1:10514 :::*
Here are the versions of the softs I am using:
Ubuntu 14.04.4
rsyslog 7.4.4
logstash 2.2.4
elasticsearch 2.3.3
That was a good thing to test, I get nothing with "netcat -ulv 10514" so it looks like it might be a rsyslog issue/misconfiguration after all but I still don't see what is wrong since the configuration remain the same as the one in the tutorial.
Just in case I checked iptables to see if it wasn't configured to block some ports but everything is fine:
$ sudo iptables -S
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
Sniffing the what is happening on the 10514 port was indeed the way to find what was wrong ! It seems that "localhost" got translated into the IPv6 address instead of the IPv4 one which ended up with the packets being dropped:
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.