# Efficient aggregation of counting nested field values

**URL:** <https://discuss.elastic.co/t/efficient-aggregation-of-counting-nested-field-values/214869>\
**Category:** Kibana\
**Created:** [January 13, 2020, 4:31pm UTC](https://discuss.elastic.co/t/efficient-aggregation-of-counting-nested-field-values/214869 "2020-01-13T16:31:56Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Elaak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/elaak/32/55014_2.png) [@Elaak](https://discuss.elastic.co/u/Elaak)\
**Post date:** [January 13, 2020, 4:31pm UTC](https://discuss.elastic.co/t/efficient-aggregation-of-counting-nested-field-values/214869/1 "2020-01-13T16:31:56Z")

</div>

Like so many others I run into the well-known "too\_many\_buckets\_exception". I am creating a data table visualization which sorts my data over four different fields, and then finally does a count over a fifth field.

A picture is worth a thousand words ...

![datatable_sketch](https://us1.discourse-cdn.com/elastic/original/3X/9/7/97036463bd79032aa19e5a737ae7120556739e9d.png)  
Because I do a nested aggregation first over my four fields, and then a combined aggregation of the results, I will quickly get too many buckets. Is there a more efficient way to aggregate my results to create the table I want?

I think the biggest complication is that the final two fields are both date fields ...

---

<div class="post-metadata">

**Author:** ![anelson-edge](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anelson-edge/32/52865_2.png) [@anelson-edge](https://discuss.elastic.co/u/anelson-edge)\
**Post date:** [January 13, 2020, 5:03pm UTC](https://discuss.elastic.co/t/efficient-aggregation-of-counting-nested-field-values/214869/2 "2020-01-13T17:03:00Z")

</div>

I was unable to find sufficient documentation for Data Table to suit my needs, but you might try a solution that seemingly worked for me. To wit, use the "Panel Option" -\> DateTimeRange -\> "Entire Time Range" to remove that error.

My hypothesis is that the other choice ("Last Value") uses an outer bucket by timestamp which doesn't work if you have a lot timestamps.

---

<div class="post-metadata">

**Author:** ![anelson-edge](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anelson-edge/32/52865_2.png) [@anelson-edge](https://discuss.elastic.co/u/anelson-edge)\
**Post date:** [January 13, 2020, 5:04pm UTC](https://discuss.elastic.co/t/efficient-aggregation-of-counting-nested-field-values/214869/3 "2020-01-13T17:04:14Z")

</div>

And @jportner suggested in another topic that he would pass along my request for improved documentation and more examples for TSVB Data Table. 🙂

---

<div class="post-metadata">

**Author:** ![Elaak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/elaak/32/55014_2.png) [@Elaak](https://discuss.elastic.co/u/Elaak)\
**Post date:** [January 14, 2020, 9:14am UTC](https://discuss.elastic.co/t/efficient-aggregation-of-counting-nested-field-values/214869/4 "2020-01-14T09:14:17Z")

</div>

Thanks for the suggestions @anelson-edge, I however do not see the options you are referring to. Are we using different elastic stack versions? Mine is `7.4.2`.

Also I do not think it will help in my case. I assume what you mean is to not aggregate over the date field, but combine all data there into one bucket. That is however not what I want, I want to be able to aggregate with separate buckets for both my date fields.

---

<div class="post-metadata">

**Author:** ![anelson-edge](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anelson-edge/32/52865_2.png) [@anelson-edge](https://discuss.elastic.co/u/anelson-edge)\
**Post date:** [January 14, 2020, 3:20pm UTC](https://discuss.elastic.co/t/efficient-aggregation-of-counting-nested-field-values/214869/5 "2020-01-14T15:20:54Z")

</div>

![Screen Shot 2020-01-14 at 10.18.31 AM](https://us1.discourse-cdn.com/elastic/original/3X/0/f/0fd9afcd9ad686ec2e63a7455120b9ee54d04755.png)

---

<div class="post-metadata">

**Author:** ![anelson-edge](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anelson-edge/32/52865_2.png) [@anelson-edge](https://discuss.elastic.co/u/anelson-edge)\
**Post date:** [January 14, 2020, 3:21pm UTC](https://discuss.elastic.co/t/efficient-aggregation-of-counting-nested-field-values/214869/6 "2020-01-14T15:21:09Z")

</div>

![Screen Shot 2020-01-14 at 10.19.47 AM](https://us1.discourse-cdn.com/elastic/original/3X/3/1/318869f8dfd620d020bd1fc5c526c20e2966df5c.png)

---

<div class="post-metadata">

**Author:** ![anelson-edge](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anelson-edge/32/52865_2.png) [@anelson-edge](https://discuss.elastic.co/u/anelson-edge)\
**Post date:** [January 14, 2020, 3:27pm UTC](https://discuss.elastic.co/t/efficient-aggregation-of-counting-nested-field-values/214869/7 "2020-01-14T15:27:01Z")

</div>

I'm using 7.4.2 as well.  
I haven't had a chance to think about your example, but with the most cursory amount of thinking, I would start with doing fields 1-4 and making sure that works as intended. You can then do sibling aggs of 1-4 in your 5th column and then do a bucket agg on the values of those 4 fields.

Perhaps a small dataset with expected results using direct ES queries and _then_ asking how to visualize would also help.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 11, 2020, 3:27pm UTC](https://discuss.elastic.co/t/efficient-aggregation-of-counting-nested-field-values/214869/8 "2020-02-11T15:27:03Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
