How can I visualize aggregation buckets using vega (not vega-lite)

Elastic Stack Kibana
1

Elasticsearch queries return an array named "buckets".
All the examples I found for vega assume that the array is named "values.

Try this gist:

2

Hi @anelson-edge

The examples you found work with inline data https://vega.github.io/vega-lite/docs/data.html#inline which is specified by a "name" and a "values" prop. Fetching data from elasticsearch works a bit differently, because the data is not included in the vega spec itself but fetched from the server when a user navigates to the page.

This blog article https://www.elastic.co/blog/getting-started-with-vega-visualizations-in-kibana goes into depth how you can do that to get the chart you want.

In your specific case the inline data specifying "values" directly would be replaced with an url like this:

{
      "name": "aggregations",
      "url": {
        "%context%": true,
        "%timefield%": "@timestamp",
        "index": "<YOUR_INDEXNAME_HERE>",
        "body": {
          "aggs": {"terms": {"terms": {"field": "<YOUR_FIELDNAME_HERE>", "min_doc_count": 0}}},
          "size": 0
        }
      },
      "format": {"property": "aggregations.terms.buckets"}
    }

Note that the "format" is part of the data source and specifies the path were the data returned from the url will be

3

Thank you for the links.
I had already seen both of the pages you linked to, and I still have some newbie questions.
AFAICT, format only applies to vega-lite, and not vega/v3. Is that right?
If that is not right, then can you explain to me how to change the kibana-vega I posted so that it works?

I tried to make a self-contained example here:


Since I'm having such troubles making the data display, can you give me full working example based on the gist I posted one line above this one?

4

Indexing the data you gave me the following spec shows a chart for me:

{
  "$schema":"https://vega.github.io/schema/vega/v3.json",
  
  
  "data": 
    [ {
      "name": "aggregations",
      "url": {
        "%context%": true,
        "index": "alerts*",
        "body" : {
          "size" : 0,
          "aggs": {
            "by_name": {
              "terms": {
                "field": "name"
              }
            }
          }
        }
      },
       format: { property: "aggregations.by_name.buckets" }
    } ],
 



  "scales": [
    {
      "name": "yscale",
      "type": "linear",
      "zero": true,
      "domain": {"data": "aggregations", "field": "doc_count"},
      "range": "height"
    },
    {
      "name": "xscale",
      "type": "band",
      "domain": {"data": "aggregations",  "field": "key"},
      "range": "width",
      "padding": 0.05
    }
  ],
  
   "marks": [ {
    "type": "rect",
    "from": { "data": "aggregations" },
    "encode": {
      "update": {
        "x":     {"scale": "xscale", "field": "key"},
        "width": {"scale": "xscale", "band": 1},
        "y":     {"scale": "yscale", "field": "doc_count"},
        "y2":    {"scale": "yscale", "value": 0}
      }
    }
  } ]
}

I changed 3 things:

  • pull format into the data source itself
  • The path in the JSON response contains by_name, because that's the name of the aggregation in the request - that was missing in property in format
  • Remove the timefield filter from the query because the alerts index doesn't have a time field
5

I copy-pasted your vega into my kibana and nothing displays.
I'm using Kibana 6.7.
Are you able to try your viz in Kibana 6.7?
And let me tell you again, how much I appreciate your help!

6

Hi @anelson-edge

I forgot to mention one detail, sorry - if I follow your script in the GH gist, I get an error because name is not indexed as a keyword which has to be the case to execute terms aggregations on it. Either switching the mapping to make name a keyword field or using name.keyword directly instead works for me. This is the spec with name.keyword:

{
  "$schema":"https://vega.github.io/schema/vega/v3.json",
  
  
  "data": 
    [ {
      "name": "aggregations",
      "url": {
        "%context%": true,
        "index": "alerts*",
        "body" : {
          "size" : 0,
          "aggs": {
            "by_name": {
              "terms": {
                "field": "name.keyword"
              }
            }
          }
        }
      },
       format: { property: "aggregations.by_name.buckets" }
    } ],
 



  "scales": [
    {
      "name": "yscale",
      "type": "linear",
      "zero": true,
      "domain": {"data": "aggregations", "field": "doc_count"},
      "range": "height"
    },
    {
      "name": "xscale",
      "type": "band",
      "domain": {"data": "aggregations",  "field": "key"},
      "range": "width",
      "padding": 0.05
    }
  ],
  
   "marks": [ {
    "type": "rect",
    "from": { "data": "aggregations" },
    "encode": {
      "update": {
        "x":     {"scale": "xscale", "field": "key"},
        "width": {"scale": "xscale", "band": 1},
        "y":     {"scale": "yscale", "field": "doc_count"},
        "y2":    {"scale": "yscale", "value": 0}
      }
    }
  } ]
}
7

I forgot to mention that I was radio silent during my vacation last week.
For reasons that I don't understand (same browser, same Kibana instance),
my viz/your-viz are now working for me.
Thanks again for helping me out.

8

And I should've said so in my original gist, but I have a default mapping in my Kibana instance that maps all strings to keywords so that was not the problem either. Hmm.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.