Kibana TSVB - Percentage of Samples crossing Threshold Filter Ratio Always Returning 0 for one Index

[shgpde (Seán)]

Hi,

First time poster, I'm having difficulty getting the data I want from a Kibana visualisation and I'm hoping for some insight.

I'm using Kibana v7.6.1 and have been running into an issue trying to use a Filter Ratio with a specific index where the ratio always evaluating to zero, and I was wondering if anyone had a similar experience, or had a suggestion for an alternate way to approach what I'm trying to do.

What I want is, to check the which documents in a range are numerically higher than a threshold, and express that as a percentage of all the documents in the range.

I've had success in the past using a TSVB with a Filter Ratio Aggregation. For example, as a metric I'll set it up like this; this is configured through UI but I'll use a code box to describe it.

TSVB: Metric
Aggregation: Filter Ratio
Numerator: maxCPUBusy:>80
Denominator: MaxCPUBusy:*
Metric Aggregation: Count

Using this configuration I'm able to return a percentage of documents that cross that threshold. It works great.
However with one particular index, I've found that the result returned is always 0.
I'm using the exact same configuration that's worked before, but even using a comparison looking for results over zero in the numerator does not return any values.
The only difference here, is that the name of the field has spaces in it. "Batch Rate", where other fields I've used in the past were in camel case.
I know that there's hits available, as if I make a table with two counts of documents, one with a filter applied looking for only documents breaking the threshold I get two different values. But if I use a Filter Ratio to try and put the second count over the first I get 0 each time.

Is it possible to replicate this function using the Math aggregation?
Has anyone encountered an issue where the Filter Ratio returns 0 even though there should be values?

-Seán

[Marco_Liberati]

Hi @shgpde

welcome to the Kibana community.
I've checked locally and I could reproduce your issue with a field with a space in it.
I've found a workaround by removing the : in the :> numerator string:

spaced field > 0

seems to work in recent Kibana TSVB versions.

[shgpde (Seán)]

Hi @Marco_Liberati ,

Thanks for checking that! Knowing you could recreate it is reassuring.

I had a try with that and still turned up a similar result.

I've tried a simple count with a KQL filter in the options, and a Filter Ratio with the same calculation and am seeing different results. Allow me to show some screenshots.
In this case, both columns show different results, but both use the following

SQL Batch Rate >= 3000

I believe that in this version of Kibana, 7.6.1, Lucene is used in the actual Numerator and Denominator rather than the KQL that appears in the Filter.

Results:

Aggregation:

In this case, there are 48 records total in this index. So I'd expect the second column to resolve to 16/48 and return 0.33.

Do you know if there's an alternate approach I can use here?

Thanks again.
Regards,
Seán

[Marco_Liberati]

With the Lucene syntax I think the right filter is 'SQL Batch Rate' [3000 TO *]

[shgpde (Seán)]

Hi @Marco_Liberati

Thanks for the help, I sorted it with that.

I did run into another issue where it would return 100% regardless of what I put in, but specifying decimals and a fixed end value resolved that. As in [3000.00 TO 10000.00]. With that, it calculates the correct value.

Thanks for the help, best regards.
Seán