Display aggregated counter value per interval

[snicoll (Stéphane Nicoll)]

I have two counters for open and closes issues, something like this:

timpestamp: epoch_mill1, total: 2000, open: 200, closed: 1800
timpestamp: epoch_mill2, total: 2200, open: 250, closed: 1950
timpestamp: epoch_mill3, total: 2300, open: 150, closed: 2150
timpestamp: epoch_mill4, total: 2500, open: 200, closed: 2300

I'd like to display a line chart that shows how much were open and closed on a given period. For instance, number of issues created per month.

I've built a line chart that uses the Average aggregation and I can see a chart that keeps increasing (makes sense). I've tried to add a date histogram aggregation but I haven't managed to make it work.

Thanks!

[AClerk]

Maybe this can help

Line chart with date histogram, split series and filter

Hi, It seems clear to me that I'm obviously misunderstanding how filters are used in Kibana's visualizations and/or am using them incorrectly. Brief background. Each document contains a unique identifier and let's say two additional fields 'infectious contact'.keyword 'lhd'.keyword [infectious] I create a Y axis and use a unique count of the unique identifier (which really I suppose wouldn't have to be unique). I then create buckets and split series as per below screenshot [unfiltered] …

[snicoll (Stéphane Nicoll)]

I don't know. I haven't made progress based on that thread though.

[snicoll (Stéphane Nicoll)]

So I've made good progress using Lens and Last Value. What I am not able to do is find a way to compute the "spread". In the example above, 500 issues we created (2500 - 2000) and 500 issues where closed (2300 - 1800). What I'd like is a visualization that allow me to display that: a metric (for the whole vizualization) or a bar chart (to show that value per month, week, etc).

[ghudgins (Graham Hudgins)]

We're working on a custom formula feature that I think might help out. Can you take a look at this unreleased screenshot and see if that's what you're trying to do? This is doing the difference between the two last_value aggregations in the report and visualizing it against time as bars.

edit: or are you trying to compute how many issues got created. i.e. last_value from the previous period compared to the same value from the current period.

[snicoll (Stéphane Nicoll)]

Thanks for sharing that @ghudgins.

What I am after is a way to compute the spread, i.e. the difference between the max and min value for a field. Showing a dashboard on the last year, for instance, would give me a metric of how much issues were created, in total for that period. The formula is to take the max value (last_value) minus the min value (the first value of the period of interest).

In a similar fashion, I'd like to "bucket" that aggregation. Taking the example above, that would be a bar chart where each bar represents the number of issues that were opened per month (or per week, etc).

Back to your example, I am not looking at computing the difference between open and closed but rather the difference between the "max open" and "min open" for the period. Once I understand how to do this, I could apply it to other fields (such as number of closed issues).

[stephenb (Stephen Brown)]

Hi @snicoll

Just driving by....

What you are describing are counters (monotonically increasing numbers) so try counter rate in Lens . Very common in infra monitoring.

In order to get a rate (Open / Hour, Day Month) you have to take a derivative that is available in lens as the Function : Counter rate.

Counter rate will show you the rate of them per bucket like 1,000 per week open

So Try
Lens
Counter Rate

You will need to probably add different series because your total, open and closed are in different fields. This should be pretty straight forward. I am not sure the Normalized goes up to months

If you need more control in TSVB there is a counter rate as well that has some more flexibility for the normalized by function

If you feel that is not what you want you can also look at the Difference Function which will Calculate the difference period over period but will nor be normailzed over time. For differences I would us Difference of Max perhaps.

Give it a try ... let us know.

[snicoll (Stéphane Nicoll)]

Thanks counter rate helped but the aggregation goes to a day max. What I'd like to do is build a dashboard over the last year (for instance) and get a bar graph where each bar represents a month and shows the number of created issues that month (one bar) and the number of closed issues that month (another bar next to it). I guess using a formula for this should work? It's a math operation between the first and the last value in the bucket...

Ideally this should be dynamic the same way Kibana does for a bunch of things. So if you show a few years, that would be by quarter or something.

[ghudgins (Graham Hudgins)]

there's an actionable request for Kibana in your response, so I logged this enhancement. [Lens] Additional normalize by unit counter rate interval options · Issue #101483 · elastic/kibana · GitHub

the dynamic aspect of your request is also something to ponder (as a user changes the scale of their dashboard intervals such as these update too)

thanks for reaching out!