Port drop statistics

[blueren (bluren)]

Hello,

I'm currently using netflow plugin in my ELK stack. I was wondering if there is any way to figure out how many packets are being dropped when feeding netflow data to the 2055 port of logstash.

Thanks

[Dheeraj_Gupta (Dheeraj)]

This can be monitored at OS level. The file /proc/net/udp keeps track of current connections and drops, you can watch it to get the cumulative number of drops.
You can also see the size of recv-q and send-q using the command netstat -nau

[rcowart (Rob Cowart)]

Use netstat -su. You will get an output like this...

IcmpMsg:
    InType3: 148757
    OutType3: 546191
Udp:
    9365593 packets received
    9902 packets to unknown port received.
    1569 packet receive errors
    314418 packets sent
    1569 receive buffer errors
    0 send buffer errors
UdpLite:
IpExt:
    InOctets: 44163894429
    OutOctets: 45634974117
    InNoECTPkts: 151561037

You will see that this system has dropped 1569 of 9365593 UDP packets due to receive buffer errors (usually because the buffer is full), which is 0.02% packet loss.

Irregardless of how much tuning you do, you should not expect to achieve 0% packet loss. That will be almost impossible. However there are things that can be done to minimize it, including Linux kernel parameter tuning and Logstash tuning. This is especially important when using the Logstash Netflow or sFlow codecs, as much more processing is handled in the input, than with simple log data.

A few months ago I helped a customer go from 85% packet loss, with out-of-the-box Linux and Logstash, to 0.05%. So there is a lot of opportunity to improve on the defaults.

Rob

[blueren (bluren)]

Thanks Rob and Dheeraj!