GB/m transferred over time between hosts (internal & external)

I tried using Packetbeat to make a dashboard to show how much GB/m or TB/h transferred between various hosts in LAN and other hosts in and out, whether between internal hosts or between internal and external hosts.

When I use source.stats.net_bytes_total and dest.stats.net_bytes_total as sum metrics, I cannot convince myself about the cumulative TB/h that came up, as our WAN bandwidth has no way to be able to transfer the number of TB/h that the dashboard shows.

What would be the fields to use to show MB/GB/TB transferred per minute and hours?

Or, am I doing wrong to capture what I want to get using the wrong aggregation type (sum), maybe?

Thank you for your help in advance!

  • Young

Are you filtering on final:true?

If you want to aggregate sums of traffic, you need to filter on final:true, or use some other technique, so that you get only the latest update from each flow. You can disable intermediate reports by setting period: -1s.

Source: https://www.elastic.co/guide/en/beats/packetbeat/current/configuration-flows.html

Hi Andrew,

That's it. I'm now convinced with the bytes transferred after adding that filter to use final:true.

So, just to confirm, if final:false, I guess it's cumulative from the very beginning of the time when the Packetbeat started collecting the packets and if final:true, only for the time duration specified (e.g., last 4 hours, last 12 hours, etc.)?

Thank you very much!

  • Young

If you do not filter out events where final: false then you are summing up the amount of bytes contained in each flow update. And the number of bytes contained in each update is a summation since the flow started. For example:

Flow Event 1: {final: false, bytes: 10}
Flow Event 2: {final: false, bytes: 20}
Flow Event 3: {final: true, bytes: 40}

If you fail to filter out the flow updates you will see 70 bytes which is wrong because the total amount is 40 bytes.

1 Like

Thank you again, Andrew!

  • Young

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.