Calculating percentage in visualization per a specific field and not the regular one

Greetings! Apologies for the subject, I could not find a better way to summerize my question.

I'm using Kibana for analysis, and I have tried to create a visualization that shows the percentage of each HTTP method per domain, at each bucket of time.
For example, let's say in domain "a[.]com" the traffic is 80% GET method and 20% POST, and in "b[.]org" it is 50%-50% at 12:00. I would love to see it in a visualization with 2 graphs (I only have GET and POST available) - one for each method, and within each graph see the percentage of the HTTP method for that domain.

I tried to do so by using split charts to the HTTP method and then split series for domain, however unfortunately the percentage calculation is wrong, as it calculates the percentage of each method against other domains and not against the other HTTP method within the specific domain:


As you can see in this example, the pink bar has around 40% GET most of the time and 10% POST which doesn't make sense (as these are the only 2 options, other methods are not logged).

For example my current visualization can tell that

  1. 'a[.]com' has 25% GET, 'b[.]org' has 75% GET
  2. 'a[.]com' has 40% POST, 'b[.]org' has 60% GET
    The total percentage of 'a[.]com' is 65% which is not what I expect, because it calculates the percentage against the other domains and not against the other methods.

This is the query I believe:

"aggs": {
    "2": {
      "date_histogram": {
        "field": "@timestamp",
        "fixed_interval": "2m",
        "time_zone": "Asia/Jerusalem",
        "min_doc_count": 1
      },
      "aggs": {
        "3": {
          "terms": {
            "field": "http_method",
            "order": {
              "_count": "desc"
            },
            "size": 2
          },
          "aggs": {
            "4": {
              "terms": {
                "field": "domain",
                "order": {
                  "_count": "desc"
                },
                "size": 5
              }
            }
          }
        }
      }
    }
  }

Hi @Halfon

welcome to the Kibana community.
You can produce this type of chart with Lens, the only limit is to create two distinct panels instead of a split chart (small multiple is a feature not yet available in Lens).

Here's an example using response.keyword: "200" as filter:

In the Kibana documentation there also other examples of ratio/percentage based charts with Lens: Analyze time series data | Kibana Guide [8.7] | Elastic

Thanks @Marco_Liberati !
Unfortunately Lens doesn't seem to be available in my organization, and won't be in the near future :frowning:
Is Lens the only option? We can't do it in the regular visualization?

Hi @Halfon Welcome to the community.

What version of the Elastic / Kibana are you on?

Hi @stephenb ,
It appears to be 7.10.2 (based on a DevTools request to /).

@Halfon

The visualization capabilities have improved immensely since 7.10 as well as many many other improvements ... perhaps you should consider upgrading..

I got this to perhaps work but I don't have 7.10 to test on

Vertical Bar Chart

X-Axis Date Histogram
Y Axis: Count
Split Series: Terms : Respons Code
Split Chart Rows: Tag (this would be your POST / GET)
Axis: y-Axis Percentage

The Chart is Broken into Rows yours would be POST / GET
Each Row is a Percentile of the Response Code for the Row (POST / GET)

Best I could do :slight_smile:

Thank you very much for all the effort @stephenb!
Unfortunately it doesn't fit my needs as I need the metric count to be normal (instead of stacked) and the percentage to be calculated per domain and not against the other domains. I guess I will just have to find other solutions. But again thanks for all the effort!!

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.