Kibana Visualization of bucket aggregation data

[jos_nubel007]

I created an index that collects metrics for different applications and I visualize these metrics in Kibana. However, there is a field that contains bucket aggregation datas (something like that: Variable width histogram aggregation). For better undestanding I show an example of a hit:

{
  "_index": "metrics-monitoring",
  "_id": "1",
  "_version": 1,
  "_score": null,
  "_source": {
    "kind": "HISTOGRAM",
    "buckets": [
      {
        "min": -3.4028234663852886e+38,
        "max": 0,
        "count": 20
      },
      {
        "min": 0,
        "max": 5,
        "count": 9
      },
      {
        "min": 5,
        "max": 10,
        "count": 1
      },
      {
        "min": 10,
        "max": 25,
        "count": 0
      },
      {
        "min": 25,
        "max": 50,
        "count": 0
      },
      {
        "min": 50,
        "max": 75,
        "count": 0
      },
      {
        "min": 75,
        "max": 100,
        "count": 0
      },
      {
        "min": 100,
        "max": 250,
        "count": 0
      },
      {
        "min": 250,
        "max": 500,
        "count": 0
      },
      {
        "min": 500,
        "max": 750,
        "count": 0
      },
      {
        "min": 750,
        "max": 1000,
        "count": 0
      },
      {
        "min": 1000,
        "max": 2500,
        "count": 0
      },
      {
        "min": 2500,
        "max": 5000,
        "count": 0
      },
      {
        "min": 5000,
        "max": 7500,
        "count": 0
      },
      {
        "min": 7500,
        "max": 10000,
        "count": 0
      },
      {
        "min": 10000,
        "max": 3.4028234663852886e+38,
        "count": 0
      }
    ],
    "count": 349
}

I would like to create a histogram/bar chart type visualization from the buckets field values. Something like this just in Kibana:

How can I do this in Kibana? Is there any way to do that?

[Andrew_Tate (Drew Tate)]

Hi @jos_nubel007 !

This is outside of how we usually expect data to be structured for visualizing in Kibana. We generally assume that each Elasticsearch document is a single datum and that Elasticsearch will be performing the aggregations. Here, it looks like you're performing some aggregations outside of Elasticsearch and storing them in a single document.

You'd definitely need to do a custom visualization (using Vega). Here are the docs for that: Vega | Kibana Guide [8.5] | Elastic

I think I got pretty close to what you want with this config.

You could also consider restructuring your data one datum to one document and rely on Elasticsearch to aggregate. Doing so would unlock the full power of our analytics tools.

[jos_nubel007]

Hi @Andrew_Tate !
Thank you very much for your answer! Your solution looks very good, this is the kind of visualization I wanted.
I have improved your code and made it dynamic. I selected an index and made the visualisation based on the last hit. I show my Vega code below:

{
  "$schema": "https://vega.github.io/schema/vega-lite/v4.17.0.json",
  "description": "A simple bar chart with embedded data.",
  data: {
    url: {
      
      
      index: metrics-monitoring*
      "body": {
        size: 1,
        "sort": [{ "time": { "order": "desc" } }],
        _source: ["buckets"]
        "query": {
          "bool": {
      
            "filter": [{"term": {"name.keyword": "http.client.duration"}}]}}}
    }
    # We only need the content of hits.hits array
    format: {property: "hits.hits"}
      
    
  }
  "transform": [
    
    {"calculate": "datum._source.buckets", "as": "buckets"},
    {"flatten": ["buckets"]},
    {"calculate": "datum.buckets.min", "as": "min"},
    {"calculate": "datum.buckets.max", "as": "max"},
    {"calculate": "datum.buckets.count", "as": "count"},
    {
      "groupby": ["min", "max"],
      "aggregate": [{"field": "count", "as": "count", "op": "sum"}]
    }
  ],
  "mark": "bar",
  "encoding": {
    "x": {"field": "min"},
    "y": {"field": "count", "type": "quantitative"}
  },
 "config": {
  "kibana": {
  "hideWarnings": true
  }
  }
}

[jos_nubel007]

@Andrew_Tate
However, there's another problem I couldn't solve.
When I try to set the "%timefield%":"date" variable it throws the following error:

url.%context% and url.%timefield% must not be used when url.body.query is set

What I would like is to always display the last item according to the filter set on the dashboard, not the globally last item of an index.

Is there any way to do that?