7.6.0 on Windows: Netflow and Syslog are not being captured by Filebeat

Elastic Stack Beats
, , ,
1

Hello! I'm looking for help on a small Docker-ised ELK instance, my filebeat inplementation isn't recognising any netflow packets from the firewalls.

I have two firewalls passing Netflow v5 (Sophos XG) and v9 (Cisco ASA) netflow to "Logger", and Logger is definitly seeing these packets on :2055 using Wireshark.
However, Filebeat doesn't seem to be seeing any of the packets, or atleast is not doing anything with them. Using netstat -ab says port UDP 2055 is used by filebeat.exe, so it has got a listener there

Setup:

  • EL&K are docker-ised, beats are not dockered
  • Filebeat is sending docs direct to Elasticsearch
  • Winlog/Metric/Heartbeat on the same host are happy
  • Ran 'setup' after configuring filebeat (no complaints)
  • Monitoring is enabled on filebeat and appearing in Kibana's stack monitoring
  • Security is enabled stack-wide, custom certificates & users

I'm using the Netflow module with the following config:

- module: netflow
  log:
    enabled: true
    var:
      netflow_host: 192.168.X.Y #Tried 0.0.0.0 and localhost
      netflow_port: 2055 #have tried other ports (2056 and 2555)
      queue_size: 32000 #tried removing this
      protocols: [ v5, v9, ipfix ] #tried removing and adding all of the options

By using -e and -d "*" I can see that it is indeed generating the monitoring documents and sending them to ES, but nothing about netflow.
It loads the module (Is "Enabled inputs: 0" normal?)

2020-02-18T07:55:46.371Z        DEBUG   [cfgfile]       cfgfile/reload.go:133   Checking module configs from: S:\beats\Filebeat/modules.d/*.yml
2020-02-18T07:55:46.372Z        DEBUG   [cfgfile]       cfgfile/cfgfile.go:193  Load config from file: S:\beats\Filebeat\modules.d\netflow.yml
2020-02-18T07:55:46.373Z        DEBUG   [cfgfile]       cfgfile/reload.go:147   Number of module configs found: 1
2020-02-18T07:55:46.376Z        INFO    crawler/crawler.go:106  Loading and starting Inputs completed. Enabled inputs: 0
2020-02-18T07:55:46.376Z        INFO    cfgfile/reload.go:171   Config reloader started
2020-02-18T07:55:46.377Z        DEBUG   [elasticsearch] elasticsearch/client.go:756     Ping status code: 200
2020-02-18T07:55:46.378Z        DEBUG   [cfgfile]       cfgfile/reload.go:197   Scan for new config files
2020-02-18T07:55:46.379Z        INFO    elasticsearch/client.go:757     Attempting to connect to Elasticsearch version 7.6.0
2020-02-18T07:55:46.380Z        DEBUG   [cfgfile]       cfgfile/cfgfile.go:193  Load config from file: S:\beats\Filebeat\modules.d\netflow.yml
2020-02-18T07:55:46.380Z        DEBUG   [elasticsearch] elasticsearch/client.go:775     GET https://192.168.X.Y:9200/_license?human=false  <nil>
2020-02-18T07:55:46.381Z        DEBUG   [cfgfile]       cfgfile/reload.go:216   Number of module configs found: 1
2020-02-18T07:55:46.381Z        DEBUG   [reload]        cfgfile/list.go:62      Starting reload procedure, current runners: 0
2020-02-18T07:55:46.382Z        DEBUG   [reload]        cfgfile/list.go:80      Start list: 1, Stop list: 0
2020-02-18T07:55:46.385Z        DEBUG   [reload]        cfgfile/list.go:101     Starting runner: netflow (log)

Then it strats listening:

2020-02-18T07:55:46.429Z        INFO    [udp]   udp/server.go:81        Started listening for UDP connection    {"address": "192.168.X.Y:2055"}

Then every so often it generated a monitoring document, the only non-monitoring (i think) log I get is:

2020-02-18T08:01:52.377Z        DEBUG   [input] input/input.go:152      Run input

This is after every monitor document, not sure if it's part of monitoring or not.

Seems like Filebeat just isn't seeing, or isn't recognising, any of the packets Logger is recieving.

So far I've tried:

  • Other ports, binding to local addresses
  • Removing various non-default values
  • Running with local admin priv
  • Using the filebeat.yml's "filebeat.inputs:" rather than netflow.yml module

I feel like this is some silly Windows thing, or something to do with stack security. If anyone has any ideas or would like some log outputs (point me where) I can oblidge. I cannot provide packet captures though.

Anything to try is welcome, I've spent about 2 hours on it so if its something simple I'll cry.

Thanks!

Update: This is the same behaviour with Syslog, packets are arriving, but filebeat isn't doing anything with them. File based logs work fine.

1 Like
2

Having similar issue on Linux, so probably not a windows thing.

1 Like
3

Turns out Syslog and UDP inputs also exhibit this behaviour, even though I can see the packets in wireshark arriving at the device and netstat says [filebeat.exe] has it's listener on UDP 514 and 2055.

For a josh I wiped filebeat and tried 7.5.2, but this didn't help.

If Logstash supported ez ECS (it will?) this wouldn't be an issue. Trying to figure out a low resource workaround for Netflow SIEM intergration atleast, since our syslog is going to use ls pipelines anyway. Will update if I find any low-effort-ish way round filebeat while supporting ECS.

4

I'am having the same issue.

Would be nice if someone shares a workaround for multiple modules in the same pattern.
Cisco Module 514 / Netflow Module 2055 --> Elasticsearch

5

I added a second NIC and used a second ip-adresse. Now it works for me.
I guess filebeat dosen't like more then one module with the same ip adresse.

1 Like
6

Thanks for updating your situation, a similar approach worked for me as well.

I set up Filebeat on a second intermediate machine and it works flawlessly now, so you're right there seems to be some issue with having multiple different ingests listening on one adapter. Troubleshooting this is far too much hastle so this will do fine.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.