Data specification help

Elastic Stack Kibana
1

i'm trying to fetch some data from my index based on the event name matching exactly certain keyword but the catch is that i need the data of the current week and the previous week and also for few more previous weeks like i want to get the percentage difference of the current week's data vs prev week's data and more likewise.
Kindly help me asap as i'm stucked in this since a week and i need to complete my dashboard asap.

Thanks in advance!

I'm adding the search query of the console which gave me the required data in response but i'm unable to get the same in the vega-lite script.
The console query is

GET my_index/_search
{
  "size": 0,
  "aggs": {
    "f": {
      "terms": {
        "field": "event.name",
        "include": [
          "match_keyword"
        ]
      },
      "aggs": {
        "filters": {
          "filters": {
            "current": {
              "key":"match_keyword",
              "range": {
                "@timestamp": {
                  "%timefilter%": true
                }
              }
            },
            "previous": {
              "range": {
                "@timestamp": {
                  "%timefilter%": true,
                  "shift": -1,
                  "unit": "week"
                }
              }
            }
          }
        }
      }
    }
  }
}

and being a begginer the vega-lite code that i've tried is this but it's not showing up anything

{
  "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
  "data": {
    "url": {
      "%context%": true,
      "index": "my_index",
      "body": {
        "aggs": {
          "f": {
            "terms": {"field": "event.name", "include": "match_keyword"},
            "filters": {
              "filters": {
                "current": {"range": {"@timestamp": {"%timefilter%": true}}},
                "previous": {
                  "range": {
                    "@timestamp": {
                      "%timefilter%": true,
                      "shift": -1,
                      "unit": "week"
                    }
                  }
                }
              }
            }
          }
        },
        "size": 0
      }
    },
    "format": {"type": "json", "property": "aggregations.f.buckets"}
  },
  "transform": [
    {
      "calculate": "if (datum.previous.doc_count, datum.current.doc_count / datum.previous.doc_count-1, null)",
      "as": "percent_diff"
    },
    {
      "calculate": "if (datum['match_keyword'].doc_count, datum['match_keyword'].doc_count, null)",
      "as": "current_result"
    }
  ],
  "vconcat": [
    {
      "title": "Percent change from 1 weeks ago",
      "width": 250,
      "height": 40,
      "mark": "text",
      "encoding": {
        "text": {"field": "percent_diff"},
        "size": {"value": 32},
        "align": {"value": "center"},
        "color": {
          "condition": {"test": "datum['percent_diff'] < 0", "value": "red"},
          "value": "green"
        }
      }
    },
    {
      "title": "Current value",
      "width": "container",
      "mark": "text",
      "encoding": {
        "text": {"field": "current_result.doc_count"},
        "fill": {"value": "black"},
        "fontWeight": {"value": "bold"},
        "align": {"value": "center"}
      }
    },
    {
      "title": "Previous value",
      "width": "container",
      "mark": "text",
      "encoding": {
        "text": {"field": "previous.doc_count", "format": ","},
        "fill": {"value": "black"},
        "fontWeight": {"value": "bold"},
        "align": {"value": "center"}
      }
    }
  ]
}
2

I don't think you need vega for that, the Lens metric in combination with formula should be able to do this. See the "Week over week" example here: Create visualizations with Lens | Kibana Guide [8.2] | Elastic

1 Like
3

Thanks for the advice flash but i have a dashboard in which i need to add this percentage difference metrics and i can't find any way/ option of lens in either way of creating new visualization or through new dashboard.

If there's another options or you can help in this then please help me out.

4

I'm not sure I understand - you can add as many Lens visualizations as you need to a dashboard.

5

yes i got your point but i need to do this in vega that's the constraint for me. Help me in vega scenario

6

Can you explain why it has to be a vega visualization?

7

That's the constraint for our team

8

I have the same constraint. Our team also need and prefer Vega visualisations because they're cleaner and neater than a grid of boxed in visualisations. I too would like to know how to solve what Vishwajeet is trying to achieve.

Dashboards aren't just for tech teams to look at. Often they're also looked at by Executives and CTOs, the very people that approve the funding for the monitoring systems we implement at our level. So making a dashboard not look like it was put together with a bunch of Lego blocks is a solid requirement that I understand.

9

Thanks for explaining - the code you paste has multiple issues (also the console query you say would give you the response you want):

  • The filters of the filters aggregation need to be wrapped into a "filters" key (in your query the outer filter is the name of the agg, the second one is the agg type, you need a third one): Filters aggregation | Elasticsearch Guide [8.2] | Elastic
  • There's no "key" query clause (and also if you have multiple you would have to wrap it into a bool query Boolean query | Elasticsearch Guide [8.2] | Elastic)
  • In the vega lite script that doesn't work you specify the terms and filters agg on the same level (there needs to be another level of "aggs")

In order to help you it's probably easiest to get started by you posting the working query (exactly how you executed it) and a description of what you want to do with it in vega - it's hard to infer the exact intent from your vega spec.

10

Dashboards aren't just for tech teams to look at. Often they're also looked at by Executives and CTOs, the very people that approve the funding for the monitoring systems we implement at our level. So making a dashboard not look like it was put together with a bunch of Lego blocks is a solid requirement that I understand.

Maybe Kibana Canvas is worth a look for your use case Canvas | Kibana Guide [8.11] | Elastic

1 Like
11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.