Help to convert hardcoded data values in Vega to ElasticSearch

Elastic Stack Kibana
1

Hi!

I have a Vega visualization working with harcoded data points. I have pasted the code in Kibana and works identically to the Vega editor.

Now I want to replace hardcoded data points with the equivalent Elasticsearch query. I have pushed those 4 datapoints as 4 documents into my Elasticsearch index, but I can't figure out how to actually make it work. I have replaced the data with the following:

  "data": {
      "name": "tree",
      "url": {
        "%context%": true,
        "index": "test",
        "body" : {
          "_source": ["p1", "p2", "p3", "size", "name"]
        }
      },

And replaced the usages of "p1", "p2" and so on with _source.p1 and so on.

However Kibana throws the error message:

Cannot read properties of undefined (reading 'name')

I don't quite understand where the problem is, could I get some help to figure out how to change the data field in Vega?

Thanks!

2

Can you post your whole spec? I am guessing where you use name field you would need to do _source.name instead. But would need to see your full spec with data.

1 Like
3

Thanks for the help and link to the docs on how to post a full spec, here it is! https://gist.githubusercontent.com/carlosgalvezp/cd2d284e5a376e9ced059e9c522475ca/raw/ff290bd8b3096d383670b54d15340a83814b6b27/test.json

4 
        "type": "formula",
        "expr": "datum.key || datum._source.name || 'src'",
        "as": "label"
      }

change to

      {
        "type": "formula",
        "expr": "datum.key || datum.name || 'src'",
        "as": "label"
      }

If you get an error like this but for size next. Find the field and remove the _source.

5

Thanks for the answer! If I do that, then no data is displayed. Isn't the problem that querying Elasticsearch puts all the datapoints into the "_source" field, and so that's why I need to type it?

6

Sorry I was just looking how to clear the error. You have an issue with your data and transformations. You have a bunch of undefined rows and x, y, and r columns are blank. This is why you aren't seeing anything.

image
image1920×1306 197 KB

Looks like your data is lacking an ID and Parent ID which is required to make this visualization work according to the Stratify Transform. Take a look at the sample and see what data they start with and what they end with. That's what you need to match.

7

Yes, the reason for that data being missing is that the "nest" transform is not working, as it's not picking up "p1", "p2" and "p3" correctly. Without Elastic, my data points look like:

{
  "p1": "a",
  "p2": "b",
  "p3": "c",
}

This works fine for the nest transform. However using Elastic, the datapoints look like:

{
  "_source": {
    "p1": "a",
    "p2": "b",
    "p3": "c",
  }
}

which makes nest no longer work.

8

I replaced mentions of p1 with _source.p1 inspired by this thread but it seems it doesn't work?

9

Understand. I am not familiar enough with the nest transformation to help out. Might be worth the time opening an issue on Vegas GitHub support page.

One thing that might help a little is the project transformation. Usually I just add that first and rename all the _source fields to drop that portion to make it easier to work with later.

I can keep looking when I get some more time though.

10

I'd try these transformations.

      "transform": [
        {
          "type": "project",
          "fields": [
            "_source.p1",
            "_source.p2",
            "_source.p3",
            "_source.size",
            "_source.name"
          ],
          "as": ["p1", "p2", "p3", "size", "name"]
        },
        {"type": "nest", "keys": ["p1", "p2", "p3"], "generate": true},
        {
          "type": "pack",
          "field": "size",
          "sort": {"field": "value"},
          "size": [{"signal": "width"}, {"signal": "height"}]
        }
      ]
11

Works like a charm, million thanks!!

1 Like
12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.