Modify Vega Json Template data source to read from Elasticsearch Index

Hello everyone,

I am new to Vega plugin in kibana and I need some help to change the json code of some visualization to read from my own Elasticsearch index. In more details, I want to use the following template:

and change the code in order to read from my own defined index, and at the end, I want to have the same results as the example shown in the link but displaying the values of three fields from my index.

My index fields that I want to show are:

  • community_label
  • group_name
  • destination_ip
  • hostname
  • application_category.

Can anybody help me to change the json code accordingly. Thanks.

Hey!
Since I don't have your original data for reference, I'm providing an example here that also uses the flare dataset to get you started.

The following bulk command can be used in Kibana's Dev Console plugin to recreate flare.json from the example within a flare Elasticearch index: https://gist.github.com/walterra/c6397055405a80b9e8ae3431d0161d4b

Once you created that index, you can use the following vega spec to create the the radial tree:

{
  "$schema": "https://vega.github.io/schema/vega/v4.json",
  "width": 720,
  "height": 720,
  "padding": 5,
  "autosize": "none",
  "signals": [
    {"name": "labels", "value": true, "bind": {"input": "checkbox"}},
    {"name": "radius", "value": 280, "bind": {"input": "range", "min": 20, "max": 600}},
    {"name": "extent", "value": 360, "bind": {"input": "range", "min": 0, "max": 360, "step": 1}},
    {"name": "rotate", "value": 0, "bind": {"input": "range", "min": 0, "max": 360, "step": 1}},
    {"name": "layout", "value": "tidy", "bind": {"input": "radio", "options": ["tidy", "cluster"]}},
    {
      "name": "links",
      "value": "line",
      "bind": {"input": "select", "options": ["line", "curve", "diagonal", "orthogonal"]}
    },
    {"name": "originX", "update": "width / 2"},
    {"name": "originY", "update": "height / 2"}
  ],
  "data": [
    {
      "name": "tree",
      "url": {
        "index": "flare",
        "body": {
          "size": 1000
        }
      },
      "format": {"property": "hits.hits"},
      "transform": [
        {"type": "stratify", "key": "_source.id", "parentKey": "_source.parent"},
        {
          "type": "tree",
          "method": {"signal": "layout"},
          "size": [1, {"signal": "radius"}],
          "as": ["alpha", "radius", "depth", "children"]
        },
        {"type": "formula", "expr": "(rotate + extent * datum.alpha + 270) % 360", "as": "angle"},
        {"type": "formula", "expr": "PI * datum.angle / 180", "as": "radians"},
        {"type": "formula", "expr": "inrange(datum.angle, [90, 270])", "as": "leftside"},
        {"type": "formula", "expr": "originX + datum.radius * cos(datum.radians)", "as": "x"},
        {"type": "formula", "expr": "originY + datum.radius * sin(datum.radians)", "as": "y"}
      ]
    },
    {
      "name": "links",
      "source": "tree",
      "transform": [
        {"type": "treelinks"},
        {
          "type": "linkpath",
          "shape": {"signal": "links"},
          "orient": "radial",
          "sourceX": "source.radians",
          "sourceY": "source.radius",
          "targetX": "target.radians",
          "targetY": "target.radius"
        }
      ]
    }
  ],
  "scales": [
    {
      "name": "color",
      "type": "sequential",
      "range": {"scheme": "magma"},
      "domain": {"data": "tree", "field": "depth"},
      "zero": true
    }
  ],
  "marks": [
    {
      "type": "path",
      "from": {"data": "links"},
      "encode": {
        "update": {
          "x": {"signal": "originX"},
          "y": {"signal": "originY"},
          "path": {"field": "path"},
          "stroke": {"value": "#ccc"}
        }
      }
    },
    {
      "type": "symbol",
      "from": {"data": "tree"},
      "encode": {
        "enter": {"size": {"value": 100}, "stroke": {"value": "#fff"}},
        "update": {
          "x": {"field": "x"},
          "y": {"field": "y"},
          "fill": {"scale": "color", "field": "depth"}
        }
      }
    },
    {
      "type": "text",
      "from": {"data": "tree"},
      "encode": {
        "enter": {"text": {"field": "_source.name"}, "fontSize": {"value": 9}, "baseline": {"value": "middle"}},
        "update": {
          "x": {"field": "x"},
          "y": {"field": "y"},
          "dx": {"signal": "(datum.leftside ? -1 : 1) * 6"},
          "angle": {"signal": "datum.leftside ? datum.angle - 180 : datum.angle"},
          "align": {"signal": "datum.leftside ? 'right' : 'left'"},
          "opacity": {"signal": "labels ? 1 : 0"}
        }
      }
    }
  ],
  "legends": [{"fill": "color", "type": "symbol", "title": "depth", "orient": "top-right"}]
}

Here's a screenshot:

Hope that helps you getting started with your own data! Let us know if you have further questions

1 Like

Thanks @walterra, I appreciate your help. In my case, I have the following dataframe that I insert in Elasticsearch:

community_label group_name application_category
0 0 Machines technology
8 2 Machines business
10 2 Machines business
30 0 Machines technology
31 0 Machines technology
33 0 Machines technology
37 0 Machines technology
49 0 Machines technology
54 0 Machines technology
57 0 Machines technology
68 0 Machines technology
75 0 Machines technology
90 0 Machines technology
91 0 Machines technology
94 0 Machines technology
95 0 Machines technology
99 0 Machines technology
100 0 Machines technology
117 0 Machines technology
129 0 Machines technology

So, I want the commulity_label field to be the first level (inner level), the group_name field to be the second level, and the application_category to be the last level (third level).

Okay, I made it work. Thanks @walterra for your help. I have just one last question; I noticed that you cannot zoom. Is there a way to add code lines in the json so that one could be able to zoom in and out the radial tree (something like two buttons for zoom in and out next to the other buttons of: labels, radius, extent, etc.). Thanks again.

Have you tried the radius slider, it should allow you to zoom (the naming is a bit misleading, it's not about the radius of the circle elements)

Would you be up to sharing your version of the vega spec here too so others can see how you adapted the queries to work with your data structure?