CPU footprint of packetbeat is high

Elastic Stack Beats
1

I am using packetbeat in af_paket traffic capturing mode, which give me about 30% CPU footprint on single core instance when I go for http and mysql monitoring only. As soon as I increase more header and cookie capture in http, CPU goes a little bit higher.

I am getting lots of "WARN Response from unknown transaction. Ignoring" warnings too, looks like packetbeat ignores already established connection's communication.

I am unable to find any documentation which can help me in setting up a dedicated server for packetbeat and how can I forward traffic from application servers to packetbeat instance.
Would be great if you guys can include this in documentation.

Packetbeat version info, I am using "1.0.0-beta3 (amd64)" on ubuntu 14.04.

Thanks,
Suraj

2

How many requests per second / packets per second are you seeing?

Indeed we should have some docs about installing a dedicate Packetbeat server. In short, I'd recommend installing a supported Linux distribution (recent ubuntu/debian or centos/redhat version) and then Packetbeat on top using af_packet.

How you forward the traffic depends on your infrastructure. Usually the easiest is to use the port mirroring functionality of your switches.

Let us know if you have specific questions.

3

Tudor,

We are running instances in AWS EC2, in terms of traffic we are getting about 100 rps.
Not sure whether port mirroring is a good Idea as it will consume extra bandwidth and doing port mirroring on EC2 is pretty complicated.
So it'll be great if we can reduce CPU footprint of Packetbeat as newrelic does it so smoothly.

Thanks,
Suraj

4

Any updates on this anyone?