Performance difference between af_packet/libpcap?

Elastic Stack Beats
1

Hi Guys,

I'm looking at the packetbeat document, and it declares af_packet has better performance than pcap.

Do we have any number regarding this performance difference?
As far as I know libpcap has leveraged af_packet in Linux. I thought they should have similar performance before.

pcap, which uses the libpcap library and works on most platforms, but it’s not the fastest option.
af__packet, which uses memory mapped sniffing. This option is faster than libpcap and doesn’t require a kernel module, but it’s Linux-specific

2

we don't have numbers and mileage might vary. Normally the performance difference is recognizable by packet loss.

But the libpcap based approach does add quite some overhead, as libpcap provides a callback based interface per packet, might enforce additional copies and most important, libpcap requires the CGO interface adding a function calling overhead per packet. af_packet on the other hand is plain GO directly accessing the in shared memory (shared with kernel) + buffer sizes are somewhat tunable.

libpcap supports SOCK_PACKET and SOCK_RAW, but it depends on compile-time flags which one is available. Plus, libpcap tries to use SOCK_RAW/DGRAM, but might silently fallback to SOCK_PACKET reading package from socket.

Using af_packet you ensure SOCK_PACKET is not used, buffers are configurable + you remove some go to C and C to go calling overhead.

3

Hi Steffens,

Thanks for your kindly reply.
I forgot the overhead of CGO. It is the killer. In my quick test, GO call C is ~500 times slower than Go call Go.

After taking a quick look at the code of libpcap (Linux), I got some minor corrections.

Regarding the additional copies, it depends on how we use it.
pcap_loop does't require additional memory copy.

And the buffer can be configured in libpcap too (pcap_set_buffer_size).

Thanks,
Wenxian

4

And the buffer can be configured in libpcap too (pcap_set_buffer_size).

Right. I have had packetbeat configuration itself in mind. In packetbeat config file you can configure the buffer size for af_packet only.

Regarding the additional copies, it depends on how we use it.
pcap_loop does't require additional memory copy.

packetbeat uses go-packet, which is not using pcap_loop, but pcap_next_ex. The mmap callback requires a memcpy to copy the packet it's content into some temporary buffer.

In addition the sniffer in packetbeat uses ReadPacketData, which also copies the packet once again (required to hold on e.g. unordered TCP packets). That is af_packet will copy the packet once, but using pcap will copy the packet twice (no matter if SOCK_RAW or SOCK_PACKET will be used).

5

Hi Steffens,
Thanks for your detailed explanation. You are quite helpful.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.