Filtered Count in Visual Builder

[Gabe.S]

Hi all, question about Visual Builder: Is there a way to apply a filter to have multiple count aggregations in a single series? I want to use a math aggregation after to compute a metric. Why? See below and please tell my how to improve my process!

Background:

I'm using Elastic Search + Kibana for logging in an application. Mostly, logs come in with a message field (something like Process A Started, Process A Finished Successfully, Process A Canceled preemptively). I'm trying to count the success rate of Process A, but ignore preemptive cancels. Note that if Process A fails we don't necessarily get a Process A failed, because the app might have crashed.

Here's the my plan:

1. Count # of messages with Process A Started
2. Count # of messages with Process A Finished Successfully
3. Count # of messages with Process A Canceled preemptively (Right now I also have to divide by 3 due to a logging error, but I'll fix that)
4. Use a math aggregation to subtract cancels from starts, and then divide successful finishes by that number to get % of non canceled Process As that finished.

I can't figure out how to do that with Visual Builder or any other visualization. Any help would be appreciated!

[lukeelmers]

Hi @Gabe.S,

You might be able to achieve what you are looking for using Timelion, with an expression like this:

.es(q="message:'Process A Started'").subtract(.es(q="message:'Process A Canceled'")).divide(.es(q="message:'Process A Finished'"))

But outside of Timelion, I'm not sure there's a way to do exactly what you're looking for.

Performing calculations on the results of multi-bucket aggregations is a use case for bucket script aggregations. These basically let you define a custom metric by using a script to perform per-bucket computations. If you combined these with filters for your specific messages, you could probably get the results you are looking for directly from Elasticsearch.

However, these are unfortunately not yet supported in Kibana. If you're interested, you can follow along with the relevant Github issue to be notified of any updates.

Thanks,

Luke