Packetbeat hardware / software requirements

hilt86 · April 30, 2021, 12:59am

I have a sensor with a i5-4590, 8GB memory running packetbeat and am still getting "dropped_because_of_gaps"

I've followed all the directions about enabling af_packet, etc yet even on a low traffic link (1-5mbps) I still get dropped events - what else can I do?

Marius_Iversen · May 1, 2021, 12:06am

Would you be able to share the packetbeat.yml configuration? Just wanted to check what settings are currently configured.

hilt86 · May 1, 2021, 12:14am

Yep can do

gist.github.com

https://gist.github.com/hilt86/590985088908b61f54de1c7d70338e6b

packetbeat.yml

###################### Packetbeat Configuration Example #######################

# This file is a full configuration example documenting all non-deprecated
# options in comments. For a shorter configuration example, that contains only
# the most common options, please see packetbeat.yml in the same directory.
#
# You can find the full configuration reference here:
# https://www.elastic.co/guide/en/beats/packetbeat/index.html

# =============================== Network device ===============================

This file has been truncated. show original

Marius_Iversen · May 1, 2021, 12:17am

I saw you mentioned that you had enabled af_packet, but it is currently disabled here: packetbeat 7.12.0 config template · GitHub

Do you know if you had tried to comment that one out? It should commonly resolve the issue.

However, if it does not work, and you are still seeing issues, could you try to set this one to false, and just see if it helps, at least then we are able to narrow it down a bit more:

gist.github.com

https://gist.github.com/hilt86/590985088908b61f54de1c7d70338e6b#file-packetbeat-yml-L62

packetbeat.yml

###################### Packetbeat Configuration Example #######################

# This file is a full configuration example documenting all non-deprecated
# options in comments. For a shorter configuration example, that contains only
# the most common options, please see packetbeat.yml in the same directory.
#
# You can find the full configuration reference here:
# https://www.elastic.co/guide/en/beats/packetbeat/index.html

# =============================== Network device ===============================

This file has been truncated. show original

hilt86 · May 1, 2021, 12:52am

sorry that was a typo - yep I've tried both pcap and af_packet and get the same result.

I want to capture flows so turning that off won't really help. How can I get to the bottom of this?

Marius_Iversen · May 1, 2021, 6:42am

So the above configuration, is af_packet currently enable, and no other fields are different from the configuration you linked?

The reason I wanted to disable it, was not to provide it as a solution, but to try to find out what might cause your issue.

Would you be able to share a logfile for example? It might include sensitive information, so if its not from a test environment of some sort, I was just wondering where you found your metrics and if you could grep the logfiles for any "ERROR" or "WARN" as well after restarting it and letting it run for a couple of minutes.

hilt86 · May 6, 2021, 6:03am

Yes that is right. Yes if I disable flows then I get no dropped_because_of_gaps errors but I can only get those if it is monitoring flows!

Here is a link to the logs packetbeat.log · GitHub

There are no errors, just a warning regarding SIP. BTW packetbeat is running in a container

hilt86 · May 13, 2021, 7:00am

any ideas @Marius_Iversen ? should I file a PR as I'd really like to get this working?

hilt86 · May 20, 2021, 11:20pm

If I am an Elastic cloud client can I get commercial support for this @Marius_Iversen ?

system · June 18, 2021, 1:20am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.