Send syslog to Filebeat server

Elastic Stack Beats
22

I confirm that filebeat is indeed listening, but it stops there. I can receive the logs just fine, but still not sending. Any other idea? Could it be a log size problem? Although I assume it would have told me in the logs..

[root@c03-logz002-01 filebeat]# netstat -tulpn | grep ':5515'
udp6       0      0 :::5515                 :::*                                2272/filebeat
[root@c03-logz002-01 filebeat]# tcpdump -i any port 5515
tcpdump: data link type LINUX_SLL2
dropped privs to tcpdump
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
08:03:26.402017 ens192 In  IP x.x.x.x.21926 > c03-logz002-01.c03.internal.5515: UDP, length 121
08:03:26.402111 ens192 In  IP x.x.x.x.21926 > c03-logz002-01.c03.internal.5515: UDP, length 98
08:03:26.411678 ens192 In  IP x.x.x.x.21926 > c03-logz002-01.c03.internal.5515: UDP, length 100
08:03:29.411817 ens192 In  IP x.x.x.x.21926 > c03-logz002-01.c03.internal.5515: UDP, length 120
08:03:30.411588 ens192 In  IP x.x.x.x.21926 > c03-logz002-01.c03.internal.5515: UDP, length 118
08:03:32.413857 ens192 In  IP x.x.x.x.21926 > c03-logz002-01.c03.internal.5515: UDP, length 109
6 packets captured
11 packets received by filter
0 packets dropped by kernel
[root@c03-logz002-01 filebeat]# tcpdump -i any port 5015
tcpdump: data link type LINUX_SLL2
dropped privs to tcpdump
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
0 packets captured
2 packets received by filter
0 packets dropped by kernel
23

Is your filebeat server able to connect to your logstash server?

Also, what does your logstash input looks like? You didn't share.

24

I do not have access to the logstash since it's from a third party. However I can reach it. I also have many other devices that can send logs just fine. At first I was trying to ship from my own logstash server (which currently sends my firewall logs successfully). After I added my switch config, the firewall logs would continue sending, but nothing from the switches. Hope this makes sense.

Initial setup : Switches > Logstash server > Third party logstash
Current setup : Switches > Filebeat server > Third party logstash

[root@c03-logz002-01 filebeat]# nc -z -v listener-ca.logz.io 5015
Ncat: Version 7.91 ( https://nmap.org/ncat )
Ncat: Connected to 99.79.1.61:5015.
Ncat: 0 bytes sent, 0 bytes received in 0.16 seconds.
25

You would need to validate on the Logstash side.

If the Filebeat port is open and your device is really sending logs to it, then you need to validate the Logstash side to see if it is correctly sending logs to Elasticsearch.

You should also validate that your switch is correctly sending logs to your filebeat by temporarily changing the logstash output to a file output.

This way you will be able to confirm that the setup Switches -> Filebeat is indeed working.

26

Very good idea, unfortunately I get no output from this. I wonder if I have some permission issues somewhere and it's not telling me...

############################# Filebeat #####################################

logging.level: info
logging.to_files: true
logging.files:
  path: /var/log/filebeat
  name: filebeat
  keepfiles: 7
  permissions: 0640

filebeat.inputs:
- type: syslog
  format: auto
  protocol.udp:
    host: "0.0.0.0:5515"

  fields:
    logzio_codec: plain
    token: xxxxxxxxxxxxxxx
    type: switch
  fields_under_root: true

#output.logstash:
#  hosts: ["listener-ca.logz.io:5015"]
#  ssl:
#    certificate_authorities: ['/etc/filebeat/AAACertificateServices.crt']

output.file:
  path: "/etc/filebeat/tmp"
  filename: filebeat
[root@c03-logz002-01 filebeat]# tcpdump -i any port 5515
tcpdump: data link type LINUX_SLL2
dropped privs to tcpdump
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
09:10:30.032588 ens192 In  IP x.x.x.x.21926 > c03-logz002-01.c03.internal.5515: UDP, length 120
09:10:34.032476 ens192 In  IP x.x.x.x.21926 > c03-logz002-01.c03.internal.5515: UDP, length 118
09:10:35.032339 ens192 In  IP x.x.x.x.21926 > c03-logz002-01.c03.internal.5515: UDP, length 109
3 packets captured
5 packets received by filter
0 packets dropped by kernel
[root@c03-logz002-01 filebeat]# ls /etc/filebeat/tmp/
[root@c03-logz002-01 filebeat]#
27

I would try to save the file outside /etc to avoid any permissions issue, maybe on /tmp.

If even that you cannot see anything in the logs, than maybe there is some issue with the sender.

What is the result if use the tcpdump with the -A parameter?

28

I did try from /tmp directly, but I was getting the same result. Here is my tcpdump:

[root@c03-logz002-01 filebeat]# tcpdump -i any port 5515 -A
tcpdump: data link type LINUX_SLL2
dropped privs to tcpdump
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
10:18:13.124869 ens192 In  IP x.x.x.x.21926 > c03-logz002-01.c03.internal.5515: UDP, length 120
E.......<..k
...
..EU......N<46>: 2023 Sep 29 10:18:17.544 EST: %AAA-6-AAA_ACCOUNTING_MESSAGE: update:10.4.0.15@pts/0:msylvestre:configure (SUCCESS)
10:18:15.124779 ens192 In  IP x.x.x.x.21926 > c03-logz002-01.c03.internal.5515: UDP, length 118
E.......<..e
...
..EU....~.E<45>: 2023 Sep 29 10:18:19.544 EST: %VSHD-5-VSHD_SYSLOG_CONFIG_I: Configured from vty by msylvestre on 10.4.0.15@pts/0
10:18:18.124684 ens192 In  IP x.x.x.x.21926 > c03-logz002-01.c03.internal.5515: UDP, length 109
E.......<..b
...
..EU....u'.<43>: 2023 Sep 29 10:18:22.544 EST: %RADIUS-3-RADIUS_ERROR_MESSAGE: RADIUS server x.x.x.x failed to respond
10:18:23.126492 ens192 In  IP x.x.x.x.21926 > c03-logz002-01.c03.internal.5515: UDP, length 109
E.......<..P
...
..EU....u'.<43>: 2023 Sep 29 10:18:27.546 EST: %RADIUS-3-RADIUS_ERROR_MESSAGE: RADIUS server x.x.x.x failed to respond
10:18:23.126596 ens192 In  IP x.x.x.x.21926 > c03-logz002-01.c03.internal.5515: UDP, length 131
E.......<..9
...
..EU.......<43>: 2023 Sep 29 10:18:27.546 EST: %RADIUS-3-RADIUS_ERROR_MESSAGE: RADIUS server x.x.x.x failed to respond evenafter all retries
10:18:23.128037 ens192 In  IP x.x.x.x.21926 > c03-logz002-01.c03.internal.5515: UDP, length 119
E.......<..B
...
..EU.....MT<43>: 2023 Sep 29 10:18:27.548 EST: %RADIUS-3-RADIUS_ERROR_MESSAGE: All RADIUS servers failed to respond after retries.
10:18:49.123507 ens192 In  IP x.x.x.x.21926 > c03-logz002-01.c03.internal.5515: UDP, length 121
E.......<...
...
..EU.......<43>: 2023 Sep 29 10:18:53.544 EST: %DAEMON-3-SYSTEM_MSG: ntp match all is not set & so packet is restricted - ntpd[4983]

7 packets captured
8 packets received by filter
0 packets dropped by kernel
29

Hello, just a quick update for those who helped me. I decided to give another try at my logstash configuration and I got it to work.

Thank you for all of your help

30

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.