Having problem getting syslog to show on Elasticsearch

Elastic Stack Logstash
1

Hi,
I'm having problems trying to get syslog sent by external sources to logstash, then to elasticsearch (Kibana). I think the problem is that I don't understand what should the host and port should be specified.

Here is my Logstash .conf

input{
syslog {
type => "syslog"
port => 5140
}
}

filter {
grok{
match => { "message" => "%{GREEDYDATA}"}
}
}

output {
elasticsearch {
hosts => ["XX.XX.XX.XX:9200"]
index => "realtime-%{+YYYY.MM.dd}"
sniffing => true
manage_template => false
}
stdout {codec => rubydebug}
}

When getting the port using $sudo lsof -i -P -n | grep LISTEN

java 1027 elasticsearch 286u IPv6 40732 0t0 TCP XX.XX.XX.XX:9300 (LISTEN)
java 1027 elasticsearch 288u IPv6 40743 0t0 TCP XX.XX.XX.XX:9200 (LISTEN)
node 1028 kibana 40u IPv4 44691 0t0 TCP XX.XX.XX.XX:5601 (LISTEN)
java 680736 logstash 133u IPv6 3221786 0t0 TCP *:9600 (LISTEN)
java 680736 logstash 139u IPv6 3217706 0t0 TCP 127.0.0.1:9601 (LISTEN)
rsyslogd 681930 syslog 7u IPv4 3232385 0t0 TCP *:514 (LISTEN)
rsyslogd 681930 syslog 8u IPv6 3232386 0t0 TCP *:514 (LISTEN)

And response from $sudo tcpdump -vv -i ens160 port 514

9:56:05.218174 IP (tos 0x0, ttl 59, id 43582, offset 0, flags [none], proto UDP (17), length 668)
XX.XX.XX.XX:XXX > elasticsearch-dev.syslog: [udp sum ok] SYSLOG, length: 640
Facility local7 (23), Severity notice (5)
Msg: date=2021-05-17 ................
0x0000: 3c31 3839 3e64 6174 653d 3230 3231 2d30
0x0010: 352d 3137 2074 696d 653d 3139 3a35 363a
0x0020: 3035 2064 6576 6e61 6d65 3d22 5650 4e2d
0x0030: 4761 7465 7761 7922 2064 6576 6964 3d22

Do you know what port/host should I use in logstash .conf file so that the syslog received is shown on Kibana (Elastic)

2

Have you tried using netcat to send some dummy data to the port to see if anything shows in stdout?

3

  1. your grok is malformed / does nothing it is missing the field

  2. what do the Logstash logs show? Do you see output from the rubydebug?

  3. Do you know the host and port for elasticsearch?

4

Thank you so much for the replies.

So I went through all discussion posssible, I found the right solution. I redirect the port from 514 to 5140 using IPTABLES.

It is now working fine. The followed problems, however, is that logstash only returns 1 incoming host IP address. But when tcpdump, it returns multiple incoming hosts. Why is that??

Thank you in advance for the answer

5

I am not clear on your question can you provide more details?

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.