Bug confirmation needed: Netflow module v7.6 doesn't support 4-7 byte pad in IPFIX template sets

[bortok (Alex Bortok)]

Can somebody confirm this is a bug before I submit it on github?

The Filebeat Netflow module ver 7.6 doesn't support 4-7 byte pad in template sets when parsing IPFIX. I have Ixia Vision E10S packet broker that can export IPFIX and the module is not compatible with the padding E10S uses for template with ID 256 - which is 4 byte long. Looking at the decoder.go code, maximum padding length allowed is 3 bytes:

140                 if buf.Len() < 4 {
141                         return templates, nil
142                 }

While RFC 7011, Section 3.3.1 allows Padding to be shorter than any allowable record in the set. The smallest allowable template record has a Record Header and at least one Field inside. This leads to minimum size of 8 bytes. Hence any bytes following a valid template in a set which are less than 8 should be unequivocally considered as pad.

I have two pcap files, one original which filebeat fails to parse, and another edited with padding removed. The original pcap causes the following error while parsing the template packet:

2020-04-20T13:29:32.605-0700 DEBUG  [netflow]              netflow/input.go:80        [ipfix] Unable to read V9 header: EOF
2020-04-20T13:29:32.605-0700 WARN   [netflow]              netflow/input.go:244     Error parsing NetFlow packet of length 138 from 172.20.100.132:47404: error reading header: EOF

While edited pcap works just fine

2020-04-20T14:51:39.583-0700 DEBUG  [netflow]              netflow/input.go:80        [ipfix] Packet from:172.20.100.132:43922 src:1 seq:438237
2020-04-20T14:51:39.583-0700 DEBUG  [netflow]              netflow/input.go:80        [ipfix] FlowSet ID 2 length 57
2020-04-20T14:51:39.583-0700 DEBUG  [netflow]              netflow/input.go:80        [ipfix] state 0xc0000c4740 addTemplate 256 0xc0000c4800
2020-04-20T14:51:39.583-0700 DEBUG  [netflow]              netflow/input.go:80        [ipfix] FlowSet ID 2 length 57
2020-04-20T14:51:39.583-0700 DEBUG  [netflow]              netflow/input.go:80        [ipfix] state 0xc0000c4740 addTemplate 257 0xc0000c4c80

Wireshark seem to have similar problem, as it interprets 4-byte padding as another template with ID=0.

Summary of my environment:

• Version: 7.6
• Operating System: Debian 10

Steps to Reproduce:

Enable netflow module

sudo filebeat modules enable netflow

Configure an available network interface with an IP used in the provided pcaps. This is what I have on my system

3: ens2f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:1e:67:6d:d0:7f brd ff:ff:ff:ff:ff:ff
    inet 172.20.100.132/24 brd 172.20.100.255 scope global dynamic ens2f1
       valid_lft 544sec preferred_lft 544sec

Configure netflow module in /etc/filebeat/modules.d/netflow.yml with the IP and port used in the provided pcaps

- module: netflow
  log:
    enabled: true
    var:
      netflow_host: 172.20.100.132
      netflow_port: 2055

Configure filebeat with console output in /etc/filebeat/filebeat.yml

output.console:
  pretty: true

Run filebeat in foreground with debug on

sudo filebeat -d netflow -e -c /etc/filebeat/filebeat.yml -path.home /usr/share/filebeat -path.config /etc/filebeat -path.data /var/lib/filebeat -path.logs /var/log/filebeat

Replay original and fixed pcap files using udpreplay (change -i parameter to the NIC with 172.20.100.132):

sudo udpreplay -i ens2f1 -l e10s.netflow.orig.pcap
sudo udpreplay -i ens2f1 -l e10s.netflow.fixed.pcap

PCAP files
e10s.netflow.orig.pcap
e10s.netflow.fixed.pcap

Thanks!
Alex