# Sum of source bytes seems impossibly large

**URL:** <https://discuss.elastic.co/t/sum-of-source-bytes-seems-impossibly-large/219961>\
**Category:** SIEM\
**Created:** [February 19, 2020, 12:26pm UTC](https://discuss.elastic.co/t/sum-of-source-bytes-seems-impossibly-large/219961 "2020-02-19T12:26:36Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![willemdh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/willemdh/32/16922_2.png) [@willemdh](https://discuss.elastic.co/u/willemdh)\
**Post date:** [February 19, 2020, 12:26pm UTC](https://discuss.elastic.co/t/sum-of-source-bytes-seems-impossibly-large/219961/1 "2020-02-19T12:26:36Z")

</div>

Hello,

Was browsing through Kibana SIEM on 7.5.2 and discovered some weird 'Bytes In', 'Bytes Out' metrics. After investigating, it seemd like some servers were sending huge amounts of traffic to my Elastic ingest nodes. I'm talking about 45 TB / 24 hours to each ingest node...

After some investigation, this data came from the flow packetbeat module:

```
packetbeat.flows:
  timeout: 30s
  period: 10s

```

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/4/b/4bc18d561e9dd41a7f465a9a59dd34458e9e1739.png)

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/8/18653dbbdf58be809873c5a8c5607829e25786a6.png)

So what is going on here? How does the Packetbeat flow functionality calculate source.bytes? The result is that Kibana network SIEM shows very weird results.. Is this a known issue?

Grtz

Willem

---

<div class="post-metadata">

**Author:** ![willemdh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/willemdh/32/16922_2.png) [@willemdh](https://discuss.elastic.co/u/willemdh)\
**Post date:** [March 5, 2020, 10:12am UTC](https://discuss.elastic.co/t/sum-of-source-bytes-seems-impossibly-large/219961/2 "2020-03-05T10:12:06Z")

</div>

Anyone?

---

<div class="post-metadata">

**Author:** ![willemdh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/willemdh/32/16922_2.png) [@willemdh](https://discuss.elastic.co/u/willemdh)\
**Post date:** [March 5, 2020, 11:29am UTC](https://discuss.elastic.co/t/sum-of-source-bytes-seems-impossibly-large/219961/3 "2020-03-05T11:29:53Z")

</div>

Same issue on 7.6.1 by the way.. Check this lol:

![image](https://us1.discourse-cdn.com/elastic/original/3X/c/5/c5edd3a3d89cf688910e051bd0288f13e653cfae.png)

This is on last 24 hours for 1 server. 1,7PB???? Unrealistic... 🙂

---

<div class="post-metadata">

**Author:** ![Mike\_Paquette](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mike_paquette/32/119011_2.png) [@Mike\_Paquette](https://discuss.elastic.co/u/Mike_Paquette)\
**Post date:** [March 5, 2020, 12:47pm UTC](https://discuss.elastic.co/t/sum-of-source-bytes-seems-impossibly-large/219961/4 "2020-03-05T12:47:10Z")

</div>

Hi @willemdh, Yes, those are some really big numbers! This could be because packetbeat reports bytes in a flow as a cumulative counts since the flow began, rather than as an incremental count since the last event. So when you sum them up, you get some huge values, the longer the flow lives, the sum (bytes) grows unexpectedly.

One option is to set `packetbeat.flows.period: -1s` so that you receive events only at the end of the flow. The byte values will then be accurate, but of course, you won't have any bytes shown for longstanding flows that have not closed yet.

Please let us know if this helps.

---

<div class="post-metadata">

**Author:** ![willemdh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/willemdh/32/16922_2.png) [@willemdh](https://discuss.elastic.co/u/willemdh)\
**Post date:** [March 6, 2020, 10:12am UTC](https://discuss.elastic.co/t/sum-of-source-bytes-seems-impossibly-large/219961/5 "2020-03-06T10:12:40Z")

</div>

@Mike_Paquette Thanks for the explanation. I will try your suggestion. It seems that the documentation indeed mentions this:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/b/5/b5824f63490500439874177da9337777a9984188.png)

So maybe the SIEM datatable containing Bytes In / Bytes Out could be tuned so it only lists the sum of traffic where flow.final equals true? That way we can still use intermediate flow reports without having this weird side effect in SIEM.

Grtz

Willem

---

<div class="post-metadata">

**Author:** ![willemdh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/willemdh/32/16922_2.png) [@willemdh](https://discuss.elastic.co/u/willemdh)\
**Post date:** [March 26, 2020, 8:40am UTC](https://discuss.elastic.co/t/sum-of-source-bytes-seems-impossibly-large/219961/6 "2020-03-26T08:40:29Z")

</div>

Setting period to -1 works. So what about:

> So maybe the SIEM datatable containing Bytes In / Bytes Out could be tuned so it only lists the sum of traffic where flow.final equals true? That way we can still use intermediate flow reports without having this weird side effect in SIEM.

---

<div class="post-metadata">

**Author:** ![Mike\_Paquette](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mike_paquette/32/119011_2.png) [@Mike\_Paquette](https://discuss.elastic.co/u/Mike_Paquette)\
**Post date:** [March 26, 2020, 10:33am UTC](https://discuss.elastic.co/t/sum-of-source-bytes-seems-impossibly-large/219961/7 "2020-03-26T10:33:07Z")

</div>

Thanks for the reminder @willemdh. Yes, I'll create an issue for tuning the queries associated with the Source IP and Destination IP tables in the Hosts views as you suggest. We should base this on ECS fields, of course 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 23, 2020, 10:33am UTC](https://discuss.elastic.co/t/sum-of-source-bytes-seems-impossibly-large/219961/8 "2020-04-23T10:33:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
