Bucket Script Aggregation per Bucket in TSVB possible?

[cyverboy2]

Hey there,

my data looks like this:

• Unique ID with should be my buckets
• Count: Number of entrys per Unique ID, each entry has a separate count value
• Max: max number of entry per Unique ID
• Min: min number of entry per Unique ID
  
  

What I want to do now is
[((Sum of Max) - (Sum of Min)] / (Sum of Count) = Desired loss rate

Is this possible in TSVB? It works for me with one Unique ID, but how can I do this with all Unique IDs?

[flash1293 (Joe Reuter)]

I'm not 100% sure what your question is, sorry If I misunderstand what you mean.

You can group by unique id using "Group By" for the series:

If you set that to "Everything", you will get the value for all ids overall.

If you want to see both at once (split by unique id plus overall values) you can clone the series one just set the "Group by" for one of them.

[cyverboy2]

Thanks for your reply and sorry for being not clear. I try to clarify it:

I want to figure out how much messages are getting lost during transmission. Therefore I want to count the incoming messages (column COUNT) compared to the expected messages. Each message per transmission session (column Unique ID) has a ascendig counter value, so I can calculate the expected messages by
Max Counter value - Min counter value = expected number of messages

The loss rate is then calculated by
1 - (Count / expected number of messages) = loss rate

In TSVB I want to visualize the loss rate over time for all Sessions, therefore I try to sum up all values for Count, Max and Min by each session. It should look like this then:

Is this possible?

Your solution works perfectly, if I want to check the loss rate per session, but I need to know the overall loss rate!

[flash1293 (Joe Reuter)]

I see, unfortunately I think you can't configure such complex aggregations in TSVB. You should be able to get the individual metrics (Sum of Count, Sum of Max and Sum of Min) in the regular visualization types by using the "Sum Bucket" aggregation, but there you can't apply the formula afterwards...

From what I can tell the only option left is to use a vega vis which allows you to write the Elasticsearch query yourself. Three "bucket sum" pipieline aggregations (https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-pipeline-sum-bucket-aggregation.html) on top of a terms agg on the id field with three nested metrics (sum, max and min) and then a bucket script aggregation applied on those bucket sums should do the trick.

Let me know if you need help with this approach, I'm sure we can get it to work.

For a TSVB solution - maybe someone else has another idea.