How to access double nested data in Kibana/Vega

Elastic Stack Kibana
1

Hi! I'm trying to display an stacked bar with information in a double nested array in Vega.
I know that people struggle with this and I've read other questions/responses, but I still cannot get this right.

This is the response that I have from ES. I need to display the frequencies, and for each frequency bar, stack the amount of responses per range:

"aggregations" : {
    "freqs" : {
      "doc_count" : 1401908921,
      "freq" : {
        "doc_count_error_upper_bound" : 16628988,
        "sum_other_doc_count" : 804305015,
        "buckets" : [
          {
            "key" : 314000.0,
            "doc_count" : 59767102,
            "rangos" : {
              "buckets" : [
                {
                  "key" : "1.0-29.0",
                  "from" : 1.0,
                  "to" : 29.0,
                  "doc_count" : 104
                },
                {
                  "key" : "30.0-34.0",
                  "from" : 30.0,
                  "to" : 34.0,
                  "doc_count" : 681
                },
                {
                  "key" : "34.0-*",
                  "from" : 34.0,
                  "doc_count" : 160348
                }
              ]
            }
          },
          {
            "key" : 266000.0,
            "doc_count" : 59764524,
            "rangos" : {
              "buckets" : [
                {
                  "key" : "1.0-29.0",
                  "from" : 1.0,
                  "to" : 29.0,
                  "doc_count" : 509
                },
                {
                  "key" : "30.0-34.0",
                  "from" : 30.0,
                  "to" : 34.0,
                  "doc_count" : 3201
                },
                {
                  "key" : "34.0-*",
                  "from" : 34.0,
                  "doc_count" : 1281864
                }
              ]
            }

So far I understand that the "property" of the format does not accept double nesting, so property: aggregations.freqs.freq.buckets works, but property: aggregations.freqs.freq.buckets.rangos.buckets does not work.

I've tried to do this with a project type such as this:

format: {
      property: aggregations.freqs.freq.buckets
    }
  }
  transform: [
    {
      type: project
      fields: [
        key
        rangos.buckets.key
        rangos.buckets.doc_count
      ]
      as: [
        frequencies
        ranges
        devices
      ]
    }
  ]

But somehow I get "undefined", so I'm clearly not doing this right.

Thanks for your support!

2

@nyuriks / @flash1293 can we please get some help?

Thanks,
Bhavya

3

With format: { property: aggregations.freqs.freq.buckets },

I suppose you may need flatten transform for rangos.buckets field to flatten the nested buckets.

To debug vega scripts, "Inspect" page will help you. You can see the intermediate results

image
image2082×855 124 KB

4

Hi!
I'm trying the flatten transform, but still get undefined for the rangos.buckets.doc_count :frowning:
I'm trying this:

 transform: [
    {
      flatten: [
        rangos.buckets
      ]
    }
    {
      calculate: datum.key
      as: frequencies
    }
    {
      calculate: datum.doc_count
      as: devices
    }
  ]
  mark: bar
  encoding: {
    x: {
      field: frequencies
      type: nominal
      axis: {
        title: false
      }
    }
    y: {
      field: devices
      type: quantitative
      axis: {
        title: Count
      }
    }
    color: {
      field: rangos.buckets.doc_count
      type: nominal
    }
  }

I'm using the inspect option for Vega but I'm unable to see what's being flattened and mapped, so I'm kind of blind on what's going wrong...

Am I using the flatten transformr right? Is there any way to see how it's being flattened, or why it's not getting the right field?

Thanks!

5

To see what I tried,

image
image1746×629 66.6 KB

If you specify "rangos.buckets.from" directly, it is undefined.

But once you calculate "rangos.buckets" field before transform, it could be defined.

image
image2321×674 110 KB

I'm not sure about the reason of this behavior..

6

Hi!
Thanks so much, it worked! The calculate+flatten allows me to access all the nested values. Now it's just a matter of playing with the fields until I plot something meaningful.

Cheers!
Inés

1 Like
7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.