Logstash is up and running:
[root@node1 logstash]# systemctl status logstash.service
● logstash.service - logstash
Loaded: loaded (/etc/systemd/system/logstash.service; enabled; vendor preset: disabled)
Active: active (running) since Sun 2021-09-26 15:48:33 CDT; 1min 40s ago
Main PID: 11288 (java)
CGroup: /system.slice/logstash.service
└─11288 /usr/share/logstash/jdk/bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djruby.compile.invokedynamic=true -Djruby.jit.threshold=0 -Djruby.regexp.interruptible=true -XX:+HeapDu...
Sep 26 15:49:22 node1.altignus.com logstash[11288]: [2021-09-26T15:49:22,285][INFO ][logstash.javapipeline ][.monitoring-logstash] Pipeline Java execution initialization time {"seconds"=>1.79}
Sep 26 15:49:22 node1.altignus.com logstash[11288]: [2021-09-26T15:49:22,519][INFO ][logstash.javapipeline ][.monitoring-logstash] Pipeline started {"pipeline.id"=>".monitoring-logstash"}
Sep 26 15:49:22 node1.altignus.com logstash[11288]: [2021-09-26T15:49:22,766][INFO ][logstash.javapipeline ][main] Pipeline Java execution initialization time {"seconds"=>1.19}
Sep 26 15:49:23 node1.altignus.com logstash[11288]: [2021-09-26T15:49:23,098][INFO ][logstash.inputs.beats ][main] Starting input listener {:address=>"0.0.0.0:5044"}
Sep 26 15:49:23 node1.altignus.com logstash[11288]: [2021-09-26T15:49:23,660][INFO ][logstash.javapipeline ][main] Pipeline started {"pipeline.id"=>"main"}
Sep 26 15:49:23 node1.altignus.com logstash[11288]: [2021-09-26T15:49:23,698][INFO ][logstash.inputs.tcp ][main][0817092fead8d45481934a588e0b833ca7514b7fd1abac1cdf6414bd0635f69e] Starting tcp input listener {:address=>"0.0.0.0:5002", :ssl_enable=>false}
Sep 26 15:49:23 node1.altignus.com logstash[11288]: [2021-09-26T15:49:23,878][INFO ][org.logstash.beats.Server][main][a853c5bafba489ac5ea10b25d4360a31917a48cb54113f43809ac0e1483dd5d1] Starting server on port: 5044
Sep 26 15:49:23 node1.altignus.com logstash[11288]: [2021-09-26T15:49:23,926][INFO ][logstash.agent ] Pipelines running {:count=>2, :running_pipelines=>[:".monitoring-logstash", :main], :non_running_pipelines=>[]}
Sep 26 15:49:23 node1.altignus.com logstash[11288]: [2021-09-26T15:49:23,959][INFO ][logstash.inputs.udp ][main][cded8a95889a97aa24e0b2f473f5bfab0f767fbcc760233b596e6f52bf1ab39f] Starting UDP listener {:address=>"0.0.0.0:5002"}
Sep 26 15:49:24 node1.altignus.com logstash[11288]: [2021-09-26T15:49:24,021][INFO ][logstash.inputs.udp ][main][cded8a95889a97aa24e0b2f473f5bfab0f767fbcc760233b596e6f52bf1ab39f] UDP listener started {:address=>"0.0.0.0:5002", :receive_buffer_bytes=>"106496", :queue_size=>"2000"}
Netstat show's the listeners on 5002 and 5044 for syslog and file beats:
[root@node1 logstash]# netstat -tauplan | grep 5002
tcp6 0 0 :::5002 :::* LISTEN 11288/java
udp 0 0 0.0.0.0:5002 0.0.0.0:* 11288/java
[root@node1 logstash]# netstat -tauplan | grep 5044
tcp6 0 0 :::5044 :::* LISTEN 11288/java
[root@node1 logstash]#
TCPDUMP shows the Syslog data hitting the interface without issue:
[root@node1 logstash]# tcpdump -vv -i eth0 port 5002 | more
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
15:52:39.058764 IP (tos 0x0, ttl 64, id 1838, offset 0, flags [none], proto UDP (17), length 244)
firewall1.altignus.com.syslog > node1.altignus.com.rfe: [udp sum ok] SYSLOG, length: 216
Facility local0 (16), Severity info (6)
Msg: 1 2021-09-26T16:52:39.066768-04:00 firewall1.altignus.com filterlog 72470 - - 4,,,1000000103,em0,match,block,in,4,0x0,,44,60538,0,none,6,tcp,44,162.142.125.90,135.23.115.184,43363,30112,0,S,2956919001,,1024,,mss
0x0000: 3c31 3334 3e31 2032 3032 312d 3039 2d32
0x0010: 3654 3136 3a35 323a 3339 2e30 3636 3736
0x0020: 382d 3034 3a30 3020 6669 7265 7761 6c6c
0x0030: 312e 616c 7469 676e 7573 2e63 6f6d 2066
0x0040: 696c 7465 726c 6f67 2037 3234 3730 202d
0x0050: 202d 2034 2c2c 2c31 3030 3030 3030 3130
0x0060: 332c 656d 302c 6d61 7463 682c 626c 6f63
0x0070: 6b2c 696e 2c34 2c30 7830 2c2c 3434 2c36
0x0080: 3035 3338 2c30 2c6e 6f6e 652c 362c 7463
0x0090: 702c 3434 2c31 3632 2e31 3432 2e31 3235
0x00a0: 2e39 302c 3133 352e 3233 2e31 3135 2e31
0x00b0: 3834 2c34 3333 3633 2c33 3031 3132 2c30
0x00c0: 2c53 2c32 3935 3639 3139 3030 312c 2c31
0x00d0: 3032 342c 2c6d 7373
15:52:39.058899 IP (tos 0x0, ttl 64, id 53902, offset 0, flags [none], proto UDP (17), length 268)
firewall1.altignus.com.syslog > node1.altignus.com.rfe: [udp sum ok] SYSLOG, length: 240
Facility local0 (16), Severity info (6)
Msg: 1 2021-09-26T16:52:39.066930-04:00 firewall1.altignus.com filterlog 72470 - - 4,,,1000000103,em0,match,block,in,4,0x0,,117,7111,0,DF,6,tcp,52,85.220.31.133,135.23.115.184,49915,57616,0,S,1388579891,,64240,,mss;nop;wscale;nop;nop;sackOK
0x0000: 3c31 3334 3e31 2032 3032 312d 3039 2d32
0x0010: 3654 3136 3a35 323a 3339 2e30 3636 3933
0x0020: 302d 3034 3a30 3020 6669 7265 7761 6c6c
0x0030: 312e 616c 7469 676e 7573 2e63 6f6d 2066
0x0040: 696c 7465 726c 6f67 2037 3234 3730 202d
0x0050: 202d 2034 2c2c 2c31 3030 3030 3030 3130
0x0060: 332c 656d 302c 6d61 7463 682c 626c 6f63
0x0070: 6b2c 696e 2c34 2c30 7830 2c2c 3131 372c
0x0080: 3731 3131 2c30 2c44 462c 362c 7463 702c
0x0090: 3532 2c38 352e 3232 302e 3331 2e31 3333
0x00a0: 2c31 3335 2e32 332e 3131 352e 3138 342c
0x00b0: 3439 3931 352c 3537 3631 362c 302c 532c
0x00c0: 3133 3838 3537 3938 3931 2c2c 3634 3234
0x00d0: 302c 2c6d 7373 3b6e 6f70 3b77 7363 616c
0x00e0: 653b 6e6f 703b 6e6f 703b 7361 636b 4f4b
15:52:39.059019 IP (tos 0x0, ttl 64, id 14314, offset 0, flags [none], proto UDP (17), length 222)
firewall1.altignus.com.syslog > node1.altignus.com.rfe: [udp sum ok] SYSLOG, length: 194
However no data is showing up in Elastic and when I do quries for SYSLOG there's nothing there.
