Difficulty in Vega with Kibana while generating a tree graph

Elastic Stack Kibana
1

I am not sure if this is the right forum to discuss problem on plugins behavior. I am using vega plugin for Kibana to generate tree type of visualization. I have data in ES like below -

Let me provide you my example in detail. I have a log file which I read using logstash and put into ElasticSearch. Some sample data stored into ElasticSearch are as below -

{
	"audit_message":"Operation_ABC",
	"step_id":4,
	"@timestamp":"2017-12-01T03:15:22.000Z",
	"exec_time":2.457235
},{
	"audit_message":"Step_DEF",
	"step_id":5,
	"@timestamp":"2017-12-01T03:15:27.084Z",
	"exec_time":0.020198,
	"parent_step_id":4
},{
       "audit_message":"Operation_GHI",
       "step_id":6,
       "@timestamp":"2017-12-01T03:15:32.175Z",
       "exec_time":0.015176,
       "parent_step_id":5
},{
       "audit_message":"Function_JKL",
       "step_id":5,
       "@timestamp":"2017-12-01T03:17:34.182Z",
       "exec_time":4.46E-4,
       "parent_step_id":4
},{
       "audit_message":"Function_MNO",
       "step_id":6,
       "@timestamp":"2017-12-01T03:17:39.254Z",
       "exec_time":1.56E-4,
       "parent_step_id":5
},{
       "audit_message":"Operation_PQR",
       "step_id":6,
       "@timestamp":"2017-12-01T03:17:44.345Z",
       "exec_time":0.003522,
       "parent_step_id":5
}

In my case, two node's a re linked with parent-child when -

  1. parent's step_id=(child's step_id)-1 [only this cond is not sufficient since there may multiple step_id with the same value; but a child can not have multiple parents]
    So, we have another cond -
  2. parent's @timestamp is the immediate previous @timestamp of the child.

I used stratify transformation on my data to give it to a tree structure. But I am struggling to provide the parentKey in stratify since I can not perform cond#2 as shown above in my vega specification. I tried lookup but, that is only meeting cond#1. Here is what I tried on my primary data source so far -

"transform": [
        {
            "type": "lookup",
            "from": "records", <----- records is secondary datasource. But that is exactly same as my primary datasource
            "key": "_source.step_id", 
            "fields": ["_source.parent_step_id"],
            "values": ["_source.@timestamp"],
            "as": ["parent_timestamp"],
            "default": null
        },
        {
          "type": "stratify",
          "key": "_source.@timestamp",
          "parentKey": "parent_timestamp"
        },
        {
          "type": "tree",
          "method": {"signal": "layout"},
          "size": [
            {"signal": "height"},
            {"signal": "width - 100"}
          ],
          "as": ["y", "x", "depth", "children"]
        }
      ]

This is creating the graph but all the child node's are pointing to one parent; since lookup is only returning the parent on the basis of step_id.

How can I query on the secondary data source with both the conditions as specified above?

2

Hi @Dimitry_Khan,

Unfortunately, we do not maintain the third-party Vega plugin. I'd recommend posting an issue on their GitHub repo.

Thanks,
Chris

3

Thanks @chrisronline.

4

@Dimitry_Khan, I see Roy just answered your question with an example at the Vega forum. Please link to it from github, as others may find that answer useful as well. Hope that was the answer you were looking for! Good luck :slight_smile:

5

@nyuriks. Sure. Working on it. Will link.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.