Visualizing a metric on two unique counts

I am collecting custom logs from a web server. I am trying to summarize what % of unique users encounter an error. I am having trouble finding a visualization that will do a % calculation on two unique (cardinatily) values.
Oversimplified version of a log entry in kibana
username: "foo@example.com", loglevel: "ERROR"
username: "bar@example.com", loglevel: "WARNING"
Right now, I can use the Metric visualizer to get the Unique count of the the number of different users that have appeared in my logs (lets say I have 50 unique users to my site), which is good. In a separate vizualizer, I can do the same thing, and add a KQL filter for loglevel:"ERROR", and i can see how many unique users encountered an error (lets say 10 users ran into errors). How can I create a vizualization box to do the 10 divided by 50 calculation and display 20%?

I tried the Filter Ratio on the TSVB, which works in doing the % calculation on other metrics, but i cannot specify Unqiue Count within the filter ratio on TSVB.

I tried the "Extended Metrics" plugin, which lets me do math on two Unique Counts, but that does not allow me to do a separate query for the errors as the denomninator

I tried to use the API and a series of aggregations to get the same numbers, but cannot fit a script into this call anywhere to do the division

> POST /index-*/_search?size=0
> {
>   "aggs": {
>     "unique_user_emails": {
>       "cardinality": {
>         "field": "useremail"
>       }
>     },
>     "found_errors": {
>       "filter": { 
>         "term": { 
>           "loglevel": "ERROR"
>         } 
>       },
>       "aggs": {
>         "uniquewitherrors": { 
>           "cardinality": { 
>             "field": "useremail" 
>           } 
>         }
>       }
>     }
>   }
>   ...
> }

Which works, and I get

> "aggregations" : {
>     "unique_user_emails" : {
>       "value" : 50
>     },
>     "found_errors" : {
>       "doc_count" : 338,
>       "uniquewitherrors" : {
>         "value" : 10
>       }
>     }
>   }

but I get errors regardless of where i try to a script to do the division

> "script": {
>     "lang": "painless",
>     "source": "found_errors>uniquewitherrors.value / unique_user_emails.value"
> }

I am beginning to think that what I am suggesting is not possible. Anyone have suggestions of other plugins or other API docs I should read to do calculations between two Unique Counts?

Which version of Kibana are you running? TSVB should support using Cardinality (which is the unique count):

Also, by default TSVB only shows the calculated value for the last time bucket, you can change that to the whole time range in the "Panel options:

2 Likes

Thanks so much for your reply!
I am running version 7.4.2. Now that I see your screenshot, I realize I was hung up on trying to do a Cardinality aggregation, and then adding an additional math aggregation, not realizing I had to select the filter ratio aggregation, and then select Cardinality as my metric aggregation. That's exactly what I was looking for

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.