Counting occurrence of two events and then displaying the percentage

Elastic Stack Kibana
1

I got through many visualizations until I came across seemingly a very simple one. I am trying various options but I feel it should not be that complicated and I am missing a trick. My ask is simple, I have Request and Response events coming in from log file and I want to show a percentage of Responses to Requests (i.e. Count of Response/Count of Requests x 100).

I added a grok filter, to tag the request and responses messages. First I used the same field name with different values. But with this I couldn't quite even calculate the sum. Then I added an integer field with value 1 for every event so that I can use 'sum'. I am able to sum the values but still not able to calculate percentages. Even the scripted field doesn't help here since it always calculates as 100% (1/1 * 100). Next I am looking at metrics filter and increment the counter through that. I should be able to achieve with this but it got me wondering whether I am making it too complex for my need. Please let me know if I am missing something here. Thanks.

1 Like
2

The Time Series Visual Builder should be helpful here.

Here is some test data I have, to try and match your data:

21 PM
Screen Shot 2017-08-07 at 2.09.21 PM.png2210×1040 196 KB

Here is my metric panel details in the Visual Builder.

06 PM
Screen Shot 2017-08-07 at 2.09.06 PM.png2202×1170 170 KB

I'm running into an issue at the moment (https://github.com/elastic/kibana/issues/13370) but I think this should get you what you want.

3

Thank You Stacey for the detailed response. Yes with this I can now show the percentage as a time series. I also wanted to show a simple metric visualization with the latest percentage value as it stands.
Is that possible?
Right now I am displaying the sum, but I really want to show only the percentage.

image

Thanks.

4

Any comments? Thanks.

5

The Visual Builder has panels for Markdown and Gauge which can show a single metric. Maybe those would work for you?

Regards,
Lee

6

So sorry for the lack of response @Nikhil_Utane.

I believe the answer is that it's not possible yet using the Time Series Visual Builder. See the response to my post in [TSVB] Metric count shows 0 · Issue #13370 · elastic/kibana · GitHub

That's a limitation of pipeline aggs, you can't use a sibling pipeline agg in a bucket_script.

Which eliminates what I was trying to do with the Visual Builder metric.

In addition, you can't use scripted fields with the Visual Builder - Scripted fields can not be used in Visual Builder · Issue #11999 · elastic/kibana · GitHub

so I don't think you can solve the issue that way either. So unfortunately, I don't think this is possible at the moment.

7

No problem Stacey.
I don't HAVE TO use Visual Builder. As long as there is some way to perform some basic computation (percentage, min/max etc), it can be through Painless, Lucene whatever option is available. Thanks.

8

Please let me know if this cannot be accomplished even outside of visual builder. Thanks.

9

I don't think there's any other way right now in Kibana besides the Visual Builder to do some calculation like this and get just the metric.

The last post here shows using the Overall Average aggregation to get a single number for your whole timespan.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.