Daily dashboard with metrics from the last document

Hello

I'm trying to use Vega as a new tool for me in Kibana. I read some basics tutorials and tried some examples with my editor. It is a powerfull tool !

I already used Kibana lens charts for networks logs with billions of documents. But I also retrieve from Elastic some homemade metrics logs with document format like this :

"_source" : {
  "count" : {
    "proc_1" : {
      "Server" : {
        "total" : 100
        "system_a" : 80
        "system_b" : 20
      },
      "BackupSite" : {
        "total" : 4
        "sys_c" : 3
        "sys_d" : 1
      }
    },
    "proc_2" : {
      "Server" : {
        "total" : 102
        "system_a" : 82
        "system_b" : 20
      },
      "BackupSite" : {
        "total" : 4
        "sys_c" : 2
        "sys_d" : 2
      }
	}
  },
  "ids" : {
    "Server" : {
      "common" : 90
      "only_in_proc_1" : 2
      "only_in_proc_2" : 8
    },
    "BackupSite" : {
      "common" : 2
      "only_in_proc_1" : 1
      "only_in_proc_2" : 1
    }
  }
}

In fact there are many many metrics in one document and I get one document each day (maybe 2 somtimes).

I would like to do a "daily" dashboard to show some metrics and compare them between proc_1 and proc_2. In a normal way, count have to be identical and only_in_proc_x have to be 0 or near zero.

I would like to use Vega to do this. So I define the data section to get only the last document of my_index like this :

  data: {
    url: {
      %context%: true
      %timefield%: @timestamp
      index: my_index
      body: {
        "size": 1, 
        "sort": [
          {
            "@timestamp": {
              "order": "desc"
            }
          }
        ],
        "_source": [ <metrics fieldname here> ]
      }
    }
    format: {property: "hits.hits"}
  }

But now I don't know what to do next. In examples, I often see data like multiple logs with a text field (here a) and a numeric field (here b).

a basic use case here :

 "data": {
    "values": [
      {"a": "A", "b": 28},
      {"a": "B", "b": 55},
      {"a": "C", "b": 43}
    ]
  }
and encoding like this
{
    "x": {"field": "a", "type": "nominal"},
    "y": {"field": "b", "type": "quantitative"}
  }

But in my case I only have <field_name> = <field_value>. I don't know how I can define differents metrics on the x axis.

Suppose I want to display count.proc_1.Server.total , count.proc_2.Server.total and difference between count.proc_1.Server.total and count.proc_2.Server.total with bars chart.

How can I do this in Vega ? Thanks for your help.

gueri

Oh oh! Maybe I found a solution with the transform "fold"

{
  "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
  "data": {
    "values": [
      {"country": "USA", "gold": 10, "silver": 20}
    ]
  },
  "transform": [{"fold": ["gold", "silver"]}],
  "mark": "bar",
  "encoding": {
    "x": {"field": "key", "type": "nominal"},
    "y": {"field": "value", "type": "quantitative"},
    "color": {"field": "key", "type": "nominal"}
  }
}

I will try with my data and let you know!

2 Likes

YES, Great! the FOLD transform was the solution.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.