Parameters in Using Calculation aggregation


(Shubham Mahajan) #1

Hi guys,
i am trying to use calculation aggregation in TSVB as below:

but is there way where i can find the number of events matching the criteria in the given timeframe using the painless script??? what other functions i can use in calculation agg.???
Please help....

i want to count the events matching (params.avg - params.std)> 0) condition??


(Peter Pisljar) #2

Can you please describe what exactly would you like to achieve there ? The current painless script will produce only 1 and 0 values for each of the buckets and it wont actually count all the documents where that expression would be true. However I don't fully understand what would you expect the average be for each document, or the standard deviation ? I think those only make sense when you are looking at buckets (multiple documents grouped together).

So at this point it looks to me this is something that you won't be able to achieve, but @simianhacker might know more ?


(Shubham Mahajan) #3

here i am comparing the data for anomalies and if the current average value passes the standard deviation value, there i want to use metric tab in TSVB for displaying status based on the passed value (1 or 0). I want to know if it's possible to calculate the no. of instances for which the anomalies occurred, basically how many times does the average value passed the std deviation value...??

Is it possible in Painless script to get the count of anomalies occurred??
@ppisljar @simianhacker please help!!


(Peter Pisljar) #4

that script will run once per bucket and produce either 0 or 1 .... if you would wish to sum all this 1s you could add another cumulative sum aggregation to sum all the 1s together which will then give you a single value.


(Peter Pisljar) #5

by the way, @simianhacker will be online in few hours


(Shubham Mahajan) #6

Thanks for the reply @ppisljar. And i tried the cumSum of calculation aggs and it gave an error.


(Chris Cowan) #7

You will want to use "Overall Sum" and then choose the calculation as the metric. That will sum up all the 1's and give you the total number of times it crossed the threshold.


(Shubham Mahajan) #8

@simianhacker Hi Chris, I tried your solution but it seems to give an error as well.


(Shubham Mahajan) #9

@simianhacker Hey Chris, can you please suggest what to do for this error in TSVB??


(Chris Cowan) #10

Which version of Kibana are you using? Does that error occur without the "Overall Sum"?

Usually that error indicates a permissions problem with the index pattern you're using.


(Shubham Mahajan) #11

hey @simianhacker, I am using 6.0.1 version. No, the error does not appear without "Overall Sum" aggs.

And now the error being displayed is a little different:

And also I tried with cumulative sum aggs


(Chris Cowan) #12

There is a bug with overall sum in 6.0.1 where it throws an error because the format of the buckets path for sibling aggs changed. It has been fixed in 6.1.1


(Shubham Mahajan) #13

@simianhacker oh okay!! but what aggregation is right for this case, cumsum or Overall sum???


(Chris Cowan) #14

Overall Sum is technically correct but cumulative sum should work, except there is a bug with cumulative sum in Elasticsearch: https://github.com/elastic/elasticsearch/issues/27544


(Shubham Mahajan) #15

@simianhacker Yeah I tried with cumsum few hours back it was working fine but after some time it keeps giving the error "The request has failed for this panel .
Also sometimes while grouping this agg on terms it gives that panel error.


(Chris Cowan) #16

Yeah.... cumulative sum is hit or miss because of that bug. 6.1.1 is pretty solid


(Shubham Mahajan) #17

oh man, we just upgraded to 6.0.1 last week. :sweat_smile:


(system) #18

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.