How to visualize tally counters (Serial Diff maybe?)

Elastic Stack Kibana
1

Using Kibana 5.4.1
I have a fairly common use case whereby servers send counter metrics at regular interval. Those counters are just a tally of some event, for example the number of connections opened, or some errors since the process started.
Those stats are meant to be compact, and while one could use 1 doc per "event", it's not the case here. So how would I go about visualizing the evolution of that counter over time?
It seems a potential candidate would be "Serial Diff" which subtracts a value at previous time from the current value. And it seems to be working as expected, but there is a problem that makes it unusable: graph always starts with a huge spike and ends with a massive drop, no matter the time window selected. It's like it's not happy with the 1st and last bucket. Any way to fix / ignore those buckets?

Example:

image
image.png2174×1517 213 KB

thanks much!

2

Have you looked into the visual builder visualization type at all? There is a serial difference option, and also an option to drop the last bucket which might help you with the drop at the end.

26%20PM
Screen Shot 2018-06-18 at 1.12.26 PM.png2464×1210 251 KB

3

Ok perfect, looks exactly like what I need, I was just delayed a few versions :slight_smile: Thanks much!

4

It looks like I will still have to wait for


I need to upgrade to have access to "drop last bucket" but even then it sounds like it may not yet drop the 1st partial bucket?

5

It looks like I am able to visualize diffs properly with following setting:

  • Metric 1: Max of the given counter
  • Metric 2: Serial diff of that max
  • Group by: server name
    It seem to be doing the job properly of grouping by time bucket / server name, then doing the diff of 1 bucket to the next. The 1st and last buckets do not seem to be an issue.

image

6

This is still not ideal, for example when a counter rolls or a server is restarted it may show huge negative values that make it harder to visualize.
I was wondering, does Timelion support doing diffs between buckets following a split?
Something like:
.es(index='myindex_*', timefield='time', split='servername:1', metric='requestcounter').serialDiff()

I guess a more generic question is: what's the best way to visualize counter type metrics with Timelion (where a counter at time T should get subtracted from T-1 to infer the change in value)?

7

Looks like this is potentially one way:

8

Got to a pretty good state using the Visual Builder (though I cannot do the ratios / diff between series like with timelion).
The trick is to add an extra FilterPositive agg as child of the SerialDiff to remove rollovers.

Last little gripe about Visual Builder: why are the same values using different colors in each graph? Anyway to fix in Kibana v5 latest? Thanks!

image
image.png4701×1259 1020 KB

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.