How to access double nested data in Kibana/Vega

Hi! I'm trying to display an stacked bar with information in a double nested array in Vega.
I know that people struggle with this and I've read other questions/responses, but I still cannot get this right.

This is the response that I have from ES. I need to display the frequencies, and for each frequency bar, stack the amount of responses per range:

"aggregations" : {
    "freqs" : {
      "doc_count" : 1401908921,
      "freq" : {
        "doc_count_error_upper_bound" : 16628988,
        "sum_other_doc_count" : 804305015,
        "buckets" : [
          {
            "key" : 314000.0,
            "doc_count" : 59767102,
            "rangos" : {
              "buckets" : [
                {
                  "key" : "1.0-29.0",
                  "from" : 1.0,
                  "to" : 29.0,
                  "doc_count" : 104
                },
                {
                  "key" : "30.0-34.0",
                  "from" : 30.0,
                  "to" : 34.0,
                  "doc_count" : 681
                },
                {
                  "key" : "34.0-*",
                  "from" : 34.0,
                  "doc_count" : 160348
                }
              ]
            }
          },
          {
            "key" : 266000.0,
            "doc_count" : 59764524,
            "rangos" : {
              "buckets" : [
                {
                  "key" : "1.0-29.0",
                  "from" : 1.0,
                  "to" : 29.0,
                  "doc_count" : 509
                },
                {
                  "key" : "30.0-34.0",
                  "from" : 30.0,
                  "to" : 34.0,
                  "doc_count" : 3201
                },
                {
                  "key" : "34.0-*",
                  "from" : 34.0,
                  "doc_count" : 1281864
                }
              ]
            }

So far I understand that the "property" of the format does not accept double nesting, so property: aggregations.freqs.freq.buckets works, but property: aggregations.freqs.freq.buckets.rangos.buckets does not work.

I've tried to do this with a project type such as this:

format: {
      property: aggregations.freqs.freq.buckets
    }
  }
  transform: [
    {
      type: project
      fields: [
        key
        rangos.buckets.key
        rangos.buckets.doc_count
      ]
      as: [
        frequencies
        ranges
        devices
      ]
    }
  ]

But somehow I get "undefined", so I'm clearly not doing this right.

Thanks for your support!

@nyuriks / @flash1293 can we please get some help?

Thanks,
Bhavya

With format: { property: aggregations.freqs.freq.buckets },

I suppose you may need flatten transform for rangos.buckets field to flatten the nested buckets.

To debug vega scripts, "Inspect" page will help you. You can see the intermediate results

Hi!
I'm trying the flatten transform, but still get undefined for the rangos.buckets.doc_count :frowning:
I'm trying this:

 transform: [
    {
      flatten: [
        rangos.buckets
      ]
    }
    {
      calculate: datum.key
      as: frequencies
    }
    {
      calculate: datum.doc_count
      as: devices
    }
  ]
  mark: bar
  encoding: {
    x: {
      field: frequencies
      type: nominal
      axis: {
        title: false
      }
    }
    y: {
      field: devices
      type: quantitative
      axis: {
        title: Count
      }
    }
    color: {
      field: rangos.buckets.doc_count
      type: nominal
    }
  }

I'm using the inspect option for Vega but I'm unable to see what's being flattened and mapped, so I'm kind of blind on what's going wrong...

Am I using the flatten transformr right? Is there any way to see how it's being flattened, or why it's not getting the right field?

Thanks!

To see what I tried,

If you specify "rangos.buckets.from" directly, it is undefined.

But once you calculate "rangos.buckets" field before transform, it could be defined.

I'm not sure about the reason of this behavior..

Hi!
Thanks so much, it worked! The calculate+flatten allows me to access all the nested values. Now it's just a matter of playing with the fields until I plot something meaningful.

Cheers!
Inés

1 Like