Network data value resets at 4GB

[patrick1 (Patrick Schlipf)]

Hei there,

I'm using Timelion to visualize the network data.
My problem is that Metricbeat will reset the value to 0 after it exceeds 4GB. To be more specific, its maximum value seems to be an unsigned Int32 in my case (even though it actually should be Int64).

Env
Windows 10
Metricbeat 5.x

Therefore it is impossible for me to visualize the network data.

---

I checked the Metricbeat debug output:

# Before
{
"@timestamp": "2017-03-13T16:37:59.853Z",
"beat": {
"hostname": "LTTW0225",
"name": "LTTW0225",
"version": "5.2.0"
},
"metricset": {
"module": "system",
"name": "network",
"rtt": 6004
},
"system": {
"network": {
"in": {
"bytes": 4030530618,
"dropped": 957,
"errors": 0,
"packets": 19277319
},
"name": "Intel(R) Ethernet Connection I218-LM",
"out": {
"bytes": 2500390842,
"dropped": 0,
"errors": 0,
"packets": 12443780
}
}
},
"type": "metricsets"
}

# After
{
"@timestamp": "2017-03-13T16:38:04.852Z",
"beat": {
"hostname": "LTTW0225",
"name": "LTTW0225",
"version": "5.2.0"
},
"metricset": {
"module": "system",
"name": "network",
"rtt": 7005
},
"system": {
"network": {
"in": {
"bytes": 179937046,
"dropped": 957,
"errors": 0,
"packets": 19570057
},
"name": "Intel(R) Ethernet Connection I218-LM",
"out": {
"bytes": 2504177322,
"dropped": 0,
"errors": 0,
"packets": 12502717
}
}
},
"type": "metricsets"
}

---

I also have tested it on multiple machines and the result was the same. The network values reset at 4GB.

Any suggestions on how to solve this problem?
I really appreciate any help.

[andrewkroh]

This seems to be a limitation in Windows. The data is returned from Windows in a MIB_IFROW which uses a DWORD (uint32) to store the value. So it will always rollover after 4GB.

I wonder if there is anything special we can do with timelion to cleanup the derivative when the rollover occurs.

[andrewkroh]

It looks like with Timelion you can apply a conditional to fix the graph.

Try adding an .if() to your expression like .derivative().if(operator="lt", if=0, then=0).

[patrick1 (Patrick Schlipf)]

Thanks for the explanation.

The fix is working as you suggested. I would have never thought about using the .if()-condition to bypass this issue. So thanks again for this workaround.