Vegalite integration with Kibana

[paul1243]

Hi,
I've tried creating a Visualization with Vega, I've the response something like below, how do I create a property for the highlighted data. Could someone please help.

"took": 765,
"timed_out": false,
"_shards": {
  "total": 215,
  "successful": 215,
  "skipped": 0,
  "failed": 0
},
"hits": {
  "total": 933,
  "max_score": 0,
  "hits": []
},
"aggregations": {
  "2": {
    "doc_count_error_upper_bound": 0,
    "sum_other_doc_count": 0,
    "buckets": [
      {
        "5": {
          "doc_count_error_upper_bound": 0,
          "sum_other_doc_count": 39,
          "buckets": [
            {
              "key": 146,                    <---
              "doc_count": 1,
              "Passed": { "value": 3 },      <---
              "total_cases": { "value": 3 }, <---
              "Failed": { "value": 0 },      <---
              "Skipped": { "value": 0 }      <---
            }
          ]
        },
        "key": "Example1",                   <---
        "doc_count": 40
      },
      {
        "5": {
          "doc_count_error_upper_bound": 0,
          "sum_other_doc_count": 48,
          "buckets": [
            {
              "key": 145,                    <---
              "doc_count": 1,
              "Passed": { "value": 2 },      <---
              "total_cases": { "value": 2 }, <---
              "Failed": { "value": 0 },      <---
              "Skipped": { "value": 0 }      <---
            }
          ]
        },
        "key": "Example2",                   <---
        "doc_count": 49
      },

[rashmi (kulkarni)]

cc @nyuriks : My colleague Yuri might be able to help you with this.

Thanks
Rashmi

[paul1243]

Sure. Thank you.

[nyuriks (Yuri Astrakhan)]

@paul1243 Sorry, I am not sure I understood. Could you sketch what you are trying to draw, e.g. a simple image, and post it here? Also, please post your ES query - in some cases it is possible to change it to simplify the graph. Thanks!

[paul1243]

Hi @nyuriks,
I'd like to create a bar stack chart, something like below:

Here is the request:

GET test_versions-*/_search
{
  "size": 0,
  "aggs": {
    "2": {
      "terms": {
        "field": "file_name.keyword",
        "size": 50,
        "order": {
          "_key": "asc"
        }
      },
      "aggs": {
        "5": {
          "terms": {
            "field": "build_number",
            "size": 1,
            "order": {
              "_key": "desc"
            }
          },
          "aggs": {
            "total_cases": {
              "sum": { "field": "total_cases" }
            },
            "Passed": {
              "sum": { "field": "Passed" }
            },
            "Failed": {
              "sum": { "field": "Failed" }
            },
            "Skipped": {
              "sum": { "field": "Aborted" }
            }
          }
        }
      }
    }
  }
}

And the response I posted on my first post. Thanks for your help.

[paul1243]

Hi @nyuriks - Could you please have a look at my post. Thanks !

[nyuriks (Yuri Astrakhan)]

@paul1243, you have several variables that you are trying to stack on top of each other, but they are all inside the same "object" (in the bucket list). Vega Lite does not support "fold" transformation yet (there is a github issues), but Vega can do it just fine. Also, you have two nested aggregations - first by file_name.keyword (top 50), followed by build_number (first one). Is that intentional? If the second aggregation is always size=1, this makes things a lot easier. In any case, here's the graph, hope that helps. Replace the values with your url to make it dynamic (I renamed 2 and 5 into easy to read names, so you might want to change your query too.

{
  "$schema": "https://vega.github.io/schema/vega/v3.json",
  "padding": 5,
  "data": [
    {
      "name": "table",
      "values": {
        "aggregations": {
          "byFilename": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
              {
                "byBuild": {
                  "doc_count_error_upper_bound": 0,
                  "sum_other_doc_count": 39,
                  "buckets": [
                    {
                      "key": 146,
                      "doc_count": 1,
                      "Passed": {"value": 3},
                      "total_cases": {"value": 3},
                      "Failed": {"value": 0},
                      "Skipped": {"value": 0}
                    }
                  ]
                },
                "key": "Example1",
                "doc_count": 40
              },
              {
                "byBuild": {
                  "doc_count_error_upper_bound": 0,
                  "sum_other_doc_count": 48,
                  "buckets": [
                    {
                      "key": 145,
                      "doc_count": 1,
                      "Passed": {"value": 2},
                      "total_cases": {"value": 2},
                      "Failed": {"value": 0},
                      "Skipped": {"value": 0}
                    }
                  ]
                },
                "key": "Example2",
                "doc_count": 49
              }
            ]
          }
        }
      },
      "format": {"property": "aggregations.byFilename.buckets"},
      "transform": [
        {
          "type": "formula",
          "expr": "datum.byBuild.buckets[0].total_cases.value",
          "as": "total_cases"
        },
        {
          "type": "formula",
          "expr": "datum.byBuild.buckets[0].Passed.value",
          "as": "Passed"
        },
        {
          "type": "formula",
          "expr": "datum.byBuild.buckets[0].Failed.value",
          "as": "Failed"
        },
        {
          "type": "formula",
          "expr": "datum.byBuild.buckets[0].Skipped.value",
          "as": "Skipped"
        },
        {
          "type": "fold",
          "fields": ["total_cases", "Passed", "Failed", "Skipped"],
          "as": ["metric", "metricValue"]
        },
        {
          "type": "stack",
          "groupby": ["key"],
          "sort": {"field": "metric"},
          "field": "metricValue"
        }
      ]
    }
  ],
  "scales": [
    {
      "name": "x",
      "type": "band",
      "range": "width",
      "domain": {"data": "table", "field": "key"}
    },
    {
      "name": "y",
      "type": "linear",
      "range": "height",
      "nice": true,
      "zero": true,
      "domain": {"data": "table", "field": "y1"}
    },
    {
      "name": "color",
      "type": "ordinal",
      "range": "category",
      "domain": {"data": "table", "field": "metric"}
    }
  ],
  "axes": [
    {"orient": "bottom", "scale": "x", "zindex": 1},
    {"orient": "left", "scale": "y", "zindex": 1}
  ],
  "legends": [{"fill": "color"}],
  "marks": [
    {
      "type": "rect",
      "from": {"data": "table"},
      "encode": {
        "enter": {
          "x": {"scale": "x", "field": "key"},
          "width": {"scale": "x", "band": 1, "offset": -1},
          "y": {"scale": "y", "field": "y0"},
          "y2": {"scale": "y", "field": "y1"},
          "fill": {"scale": "color", "field": "metric"}
        }
      }
    }
  ]
}

[paul1243]

WoW.. This is awesome @nyuriks . Thank you so much.
X-Axis Labels are coming in Horizontal, so the names of the file names are overlapping. How do I get the labels Vertically. Thanks.

[nyuriks (Yuri Astrakhan)]

You can change the angle of the axes labels. For Vega, see axes docs (search for angle). For Vega Lite, there is a labelAngle axes param.