Import large PCAP with Packetbeat

Elastic Stack Beats
1

Hey,
we sitting here in a Study-Project in Germany and want to work with packetbeat and ELK to find security issues in network-traffic. The live-capturing works fine.

Now, we want to import our old PCAP-files from the last 3 years (approx. 3GB/day). If we use
"packetbeat run -I "PCAP-FILE" -t "
we see captured packets in Kibana, but only 700 packets per 30 minutes. There should be much more! It seems that packetbeat stops after this 700 packets.

Can somebody help us to solve this problem?

Thanks!!!!

---Update ----
The only logs we see in Kibana are ICMP-Packets
If we use live-capturing, we see all the traffic. We use the same config-file.

2

Is there any difference if you add -E packetbeat.shutdown_timeout=1h to the args? This will allow it to flush all of its events to the output before exiting.

1 Like
3

Thanks for your reply! I'll try it out tomorrow. But we've found the main problem from our Import-Procedure. Packetbeat generates the flow-streams and set the timestamp with the current system-time and not the "historical"-Timestamp from the packets. Thats weird! Only ICMP-Packets have the correct timestamp. Do you have a solution for this?

1 Like
4

The timestamp issue is a known problem. This is one of the reasons why the PCAP processing is marked as only useful for testing Packetbeat itself.

We have a pending issue to improve pcap file processing to make it more useful:

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.