Packetbeat input from pcap -> all sniffed field on ELK

[mardux]

Hi all.

i try to import a pcap file in ELK, using packetbeat.

in my pcap file i have about 100k packets (999671 packet are HTTP traffic on port 80, the rest other protocol and port, like TLS/HTTPS on port 443)

using this command:

.\packetbeat.exe -e -c packetbeat.yml -t -I C:\tmp\my.pcap -d "publish"

but in kibana i see only few packets, not HTTP but other protocol.

seem that packetbeat (or kibana) filter most packets.

below an example of my event

my questions is:
how export all pcap packet on ELK without filter, and how see all filed of my events?

thanks for the support

[adrisr (Adrian Serrano)]

• Which version of packetbeat are you using? Recently we've fixed a few issues with the HTTP parsing.

• Do you see any ERROR or WARN-ing in packetbeat output?

• If you can share the pcap or some part of it, I'll take a look.

[mardux]

@mardux

w export all pcap packet on ELK without filter, and how see all filed of my events

Hi Adrian. thanks for your response

i use the lastest version, packetbeat-6.4.0-windows-x86_64.

i don'see particolar errors on the output, no warn or error, only info or debug;
for example, for a PCAP of 4 millions of packets (370 MB, mainly HTTP traffic on port 80):

for this case, 0 events uploaded on ELK. this is the output

continue later in the next post

[mardux]

another example, for a PCAP of 100k events (70 MB, mainly HTTP traffic on port 80):

in this case only 8 events was uploaded on ELK (TLS traffic on port 443). the output is little different, i see the event uploaded on the output of the command:

output without events:

thanks very much for your help

[adrisr (Adrian Serrano)]

Hi @mardux

I've analysed your pcap. There's a couple of issues:

• Most of the traffic is a DoS attack to port 80, where there isn't any HTTP protocol transaction, just a flood of random data to the http server. Packetbeat's http protocol will not report anything about this as it is not detected as valid http protocol data. To monitor this kind of packets you should use packetbeat.flows (which are currently disabled in your config).

• There are two HTTP requests (without a matching response) in the pcap file. However, these are not reported. This is caused by a known problem with pcap file ingestion (and the reason why it is marked as a testing-only feature in the docs): Packetbeat is designed to report unmatched transactions like this when a timeout occurs. If processing of the pcap file terminates before this timeout is reached, the transactions are lost. I've created an issue in our repository to work on improving these shortcomings of the pcap file processor: https://github.com/elastic/beats/issues/8255.

[mardux]

Thanks for your support Adrian.

i confirm that it's traffic of DDOS attack, with random Ack packets.

it's clear, packetbeat can report only valid session.

in any case very compliments for the ELK stack!

regards