Vega Is Not Showing Anything Despite Code Passing Error-Free

Elastic Stack Kibana
1

im unable to get my Vega to show smth?

image
image1868×764 47 KB

this the o/p using Vega-Lite 2.0

image
image1878×740 42.3 KB

this is the o/p using Vega 4.0

{
"$schema": "https://vega.github.io/schema/vega/v4.json",
"data": {
  "url": {
    "%context%": true,
    "%timefield%": "DateTime",
    "index": "nsew*",
    "body": {
      "size": 10000,
      "_source": ["Current_Platform_Name", "Align_Distance_To_Go_mm","Train_Type"]
    }
  },
  "format": {"property": "hits.hits"}
},
"transform": [
 {"filter": {"field": "datum._source['Align_Distance_To_Go_mm']", "range": [-300, 300]}},
 {
   "joinaggregate": [{
     "op": "count",
     "field": "datum.Align_Distance_To_Go_mm",
     "as": "count_all_Dist"
    }],
    "groupby": ["Current_Platform_Name"]
  },
 
  {
    "joinaggregate": [{
      "op": "count",
      "field": "datum.Align_Distance_To_Go_mm",
      "as": "count_within_Dist"
    }],
    "groupby": ["Current_Platform_Name"]
    },
   {
    "calculate": " (datum.count_within_Dist/datum.count_all_Dist)*100 ",
    "as": "Percent_Compliance"
  }
],

"layer": [
{
"mark": "bar",
"height": {"step": 5},
"encoding": {
  "x": {
    "field": "Percent_Compliance",
    "type": "quantitative",
    "axis": {"title": "Percent_Compliance"}
  },
  "y": {
    "field": "_source.Current_Platform_Name",
    "type": "nominal",
    "axis": {"title": "Platform Name"}
  },
  "color": {
    "field": "_source.Current_Platform_Name",
    "type": "nominal",
    "legend": {"title": "Platform Legend"}
  },
  "shape": {"field": "_source.Current_Platform_Name", "type": "nominal"}
}
}
]

}

here is my code. i dont get why its not working out.

i do prefer using Vega 4.3.0 - so i do seek your advice from the Vega approach instead. Thanks

2

I think your specs have multiple problems, and I will try to help you find those problems. The main problems are:

  • Transforms that use field do not use datum. You should write _source.Align_Distance_To_Go_mm instead of datum._source.Align_Distance_To_Go_mm in transforms.
  • Transforms are not the same in Vega and Vega-lite- you can't copy and paste from Vega-lite into Vega.

The best way to debug this is to use the browser dev tools and inspect the following two variables:

VEGA_DEBUG.view.data('source_0')
VEGA_DEBUG.view.data('data_0')

In vega-lite, datum refers to data_0, while source_0 is the ES data.

Putting this together, I think this is what you need to write:

"$schema": "https://vega.github.io/schema/vega-lite/v2.json",
...
"transform": [
 {"filter": {"field": "_source.Align_Distance_To_Go_mm", "range": [-300, 300]}},
 {
   "joinaggregate": [{
     "op": "count",
     "field": "_source.Align_Distance_To_Go_mm",
     "as": "count_all_Dist"
    }],
    "groupby": ["_source.Current_Platform_Name"]
  },
  {
    "joinaggregate": [{
      "op": "count",
      "field": "_source.Align_Distance_To_Go_mm",
      "as": "count_within_Dist"
    }],
    "groupby": ["_source.Current_Platform_Name"]
    },
   {
    "calculate": " (datum.count_within_Dist/datum.count_all_Dist)*100 ",
    "as": "Percent_Compliance"
  }
],
3

hi Wylie, thanks for ur explanation but hold on.

Am i only restricted to using Vega-Lite only for Kibana?

8

I believe your issue is an invalid filter. Try this:

{ "filter": "datum._source.Align_Distance_To_Go_mm >= -300 && datum._source.Align_Distance_To_Go_mm <= 300" },

This visualization should be possible.

9

image
image1346×664 50.5 KB

nope still didnt work out.

any idea why

10

Could you post the JSON code and the error messages? Can't read the screenshots

12 
// Transforms that use field do not use datum (ie joinaggregate)
// But Transforms like calculate do

{
    "$schema": "https://vega.github.io/schema/vega-lite/v2.json",
    "mark": "point",
    "data": {
      "url": {
        "%context%": true,
        "%timefield%": "Arrival_Time",
        "index": "nsew*",
        "body": {
          "size": 10000,
          "_source": ["Current_Platform_Name", "Align_Distance_To_Go_mm","Train_Type"]
        }
      },
      "format": {"property": "hits.hits"}
    },
 "transform": [
 { "filter": "datum._source.Align_Distance_To_Go_mm >= -300 && datum._source.Align_Distance_To_Go_mm <= 300" },
 {
   "joinaggregate": [{
     "op": "count",
     "field": "_source.Align_Distance_To_Go_mm",
     "as": "count_all_Dist"
    }],
    "groupby": ["_source.Current_Platform_Name"]
  },
  {
    "joinaggregate": [{
      "op": "count",
      "field": "_source.Align_Distance_To_Go_mm",
      "as": "count_within_Dist"
    }],
    "groupby": ["_source.Current_Platform_Name"]
    },
   {
    "calculate": " ( datum.count_within_Dist / datum.count_all_Dist)*100 ",
    "as": "Percent_Compliance"
  }
],
    "encoding": {
      "x": {
        "field": "Percent_Compliance",
        "type": "quantitative",
        "axis": {"title": "Percent_Compliance"}
      },
      "y": {
        "field": "_source.Current_Platform_Name",
        "type": "nominal",
        "axis": {"title": "Platform Name"}
      },
      "color": {
        "field": "_source.Current_Platform_Name",
        "type": "nominal",
        "legend": {"title": "Platform Legend"}
      },
      "shape": {"field": "_source.Current_Platform_Name", "type": "nominal"}
    }
  }
13

Okay, here is my recommendation for debugging these transformations- I think they are the cause of the issue, and you should start by removing all transformations and adding them one by one. After each transformation, inspect the VEGA_DEBUG.view.data('data_0') to see what it looks like, then iterate. Also, VEGA_DEBUG.view._runtime.data contains all the data that is generated internally, but it's a little less useful.

If you are trying to do something like "Count all the documents with this particular condition" then I actually think you should be using Elasticsearch filters instead of Vega filters.

15

Vega does have a count operator, but your visualization will be much faster if you use the Elasticsearch filters aggregation instead of looking at only 10k docs. Here is how to create a filters aggregation: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-filters-aggregation.html

Here is an example that is similar to what you're doing:

{
  $schema: https://vega.github.io/schema/vega-lite/v2.json
  data: {
    url: {
      %context%: true
      %timefield%: @timestamp
     index: metricbeat-*
     body: {
        aggs: {
          categories: {
            terms: { field: "some_category_field", size: 20 }
            aggs: {
              in_or_out: {
                filters: {
                  filters: {
                    in: {
                      range: {
                        "some_number_field": {
                          "gte": -300,
                          "lte": 300
                        }
                      }
                    },
                    out: {
                      bool: {
                        must_not: {
                          range: {
                            "some_number_field": {
                              "gte": -300,
                              "lte": 300
                            }
                          }
                        }
                      }
                    }
                  }
                }
              }
            }
          }
        }
        size: 0
      }
    }

    format: {property: "aggregations.categories.buckets"}
  }

  transform: [
    {
      calculate: "datum.in_or_out.buckets.in.doc_count / datum.in_or_out.buckets.out.doc_count",
      as: "percent"
    }
  ]

  mark: bar

  encoding: {
    y: {
      field: "key"
      type: "ordinal"
      sort: {
        field: "percent",
        order: "descending"
        op: "max"
      }
    }
    x: {
      field: "percent"
      type: quantitative
      format: ".2%"
    }
  }
}
18

Yes, it's an ES search query.

19

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.