Auto interval creates spiky graphs, 1m interval OK

Elastic Stack Kibana
1

Hi,

I'm having some problems graphing netflow bps in timelion.

Interval = auto

$q='test_name:test_1', .es($q,metric='sum:netflow.in_bytes').scale_interval(1s).divide(1024).label('Up - KBps'), $q='test_name:test_1', .es($q,metric='sum:netflow.out_bytes').scale_interval(1s).divide(1024).label('Down - KBps')

This (appears to) work but when I look at a short timeframe, lets say 15 minutes the graph is very spiky and shows too high bandwidth utilization but when I change the time frame to lets say 4 hours it looks okay.

When I change the interval to 1m it looks okay as well but the problem with that is that I can't use that because it will create too many buckets on anything over a couple of days so I want it to auto scale.

This is a screenshot from a +/- 500MB file download, downloading steady at around 1mbit. With a four hour scale the graph looks good but on a lower scale the graph is incorrect.

timelion.png1861×369 22.5 KB

edit: added screenshot

2

Perhaps using .fit(scale) or .fit(carry) function would smooth out the empty buckets the way you want?

3

.fit doesn't really change anything. There is no documentation on exactly what scale or carry is supposed to do either.

The problem is that after years of people asking Kibana/timelion still does not appear to support xxx / per second properly. Scale_interval should do that but it's not accounting for netflow records not coming in per second.

E.g. if I start a large file download I can see that every minute or so a record is created with out_bytes being 50MB and other smaller records are created as well with e.g. web browsing. The problem is that the large 50MB chunks are fed into elastic every minute and the logic timelion is applying to it is incorrect.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.