# Nested aggregation count seems incorect

**URL:** https://discuss.elastic.co/t/nested-aggregation-count-seems-incorect/49687
**Category:** Elasticsearch
**Created:** [May 10, 2016, 4:36pm UTC](https://discuss.elastic.co/t/nested-aggregation-count-seems-incorect/49687 "2016-05-10T16:36:48Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![ffrinton](https://avatars.discourse-cdn.com/v4/letter/f/e0b2c6/32.png) [@ffrinton](https://discuss.elastic.co/u/ffrinton)
#### Post date: [May 10, 2016, 4:36pm UTC](https://discuss.elastic.co/t/nested-aggregation-count-seems-incorect/49687/1 "2016-05-10T16:36:48Z")

</div>

ES v2.1.1

I have a data structure containing embedded history, where a docs history is an array of history objects, each of which contains a number of fields.

Unfortunately the avail space for an issue here is limited to 5000 chars - so I had to resort to external links! This is a real shame as these will disappear in time ...

[Map extract](http://paste.ubuntu.com/16348371/)

I need to create aggregate queries with conditions on the parent doc as well as on multiple history fields.

[Query](http://paste.ubuntu.com/16348238/)

[Query Result](http://paste.ubuntu.com/16348456/)

This query comes close to the desired result. My data contains the following:

11 docs which contain only a history entry with action\_name `implement`  
21 docs which contain only a history entry with action\_name `submit`  
23 docs which contain both an `implement` and a `submit` history entry.

Hence a search for just `submit`ed items should (and does) yield 21 + 23 = 44 hits  
A search for just `implement`ed items should (and does) yield 11 + 23 = 34 hits

However, when querying on both aggregating on action\_name as per the given query,  
the bucket doc\_counts contain 55/56 for `submit` (should be 44) , 34 for `implement` (correct).

I do not understand the 55/56 discrepancy, and also noted that 55 is actually  
the sum of all docs (11 + 21 + 23).

So it seems the count for `submit` results is actually the sum for `implement` and `submit` ??

Hopefully this is a problem in my query - can anyone point me to it?

---

<div class="post-metadata">

### Author: ![ffrinton](https://avatars.discourse-cdn.com/v4/letter/f/e0b2c6/32.png) [@ffrinton](https://discuss.elastic.co/u/ffrinton)
#### Post date: [May 13, 2016, 11:12am UTC](https://discuss.elastic.co/t/nested-aggregation-count-seems-incorect/49687/2 "2016-05-13T11:12:17Z")

</div>

And as anticipated the issue was in the query. The _entire_ nested sub-query must be repeated in the aggregation as a filter - as shown below, now results are as expected!

BTW - providing nicely formatted (indented) json here consumes all my available post-characters in white-space!

> {"query":{"filtered":{"filter":{"bool":{"must":[{"term":{"domain.acronym":"dom"}},{"bool":{"must":[{"term":{"company.name":"abc"}},{"bool":{"must":[{"term":{"severity":"important"}},{"nested":{"path":"history","inner\_hits":{"size":50,"name":"history"},"query":{"bool":{"must":[{"terms":{"history.action\_name":["implement","submit"]}},{"bool":{"must":[{"range":{"history.created\_at":{"gte":"2014-04"}}},{"range":{"history.created\_at":{"lt":"2016-07"}}}]}}]}}}}]}}]}}]}}}},"size":0,"aggs":{"company.name":{"terms":{"size":0,"field":"company.name.raw"},"aggs":{"severity":{"terms":{"size":100,"field":"severity.raw"},"aggs":{"history":{"nested":{"path":"history"},"aggs":{"by\_action":{"filter":{"bool":{"must":[{"terms":{"history.action\_name":["implement","submit"]}},{"bool":{"must":[{"range":{"history.created\_at":{"gte":"2014-04"}}},{"range":{"history.created\_at":{"lt":"2016-07"}}}]}}]}},"aggs":{"my\_name":{"terms":{"field":"history.action\_name","size":100},"aggs":{"container":{"reverse\_nested":{},"aggs":{"stats\_number":{"stats":{"field":"number"}}}}}}}}}}}}}}}}

---

<div class="post-metadata">

### Author: ![jpountz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jpountz/32/45836_2.png) [@jpountz](https://discuss.elastic.co/u/jpountz)
#### Post date: [May 13, 2016, 3:35pm UTC](https://discuss.elastic.co/t/nested-aggregation-count-seems-incorect/49687/3 "2016-05-13T15:35:12Z")

</div>

Maybe you want to use the `reverse_nested` aggregation in order to count top-level documents rather than inner documents?

---

<div class="post-metadata">

### Author: ![ffrinton](https://avatars.discourse-cdn.com/v4/letter/f/e0b2c6/32.png) [@ffrinton](https://discuss.elastic.co/u/ffrinton)
#### Post date: [May 13, 2016, 4:26pm UTC](https://discuss.elastic.co/t/nested-aggregation-count-seems-incorect/49687/4 "2016-05-13T16:26:21Z")

</div>

Thanks for your comment jpountz. Any advice is welcome! The thinking of the current query is:

Queries are built by a GUI. Queries may be aggregated or not. The first part of the query (excl the aggs) is always the same regardless whether aggregated or not. In the case where they are not - the `inner_hits` on the nested history items is used to obtain the array offsets of the history entries (needed to retrieve & display only the matched hits).

In the aggregation case there already _is_ a `reverse_nested` clause to obtain parent stats.

Did I miss something perhaps?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 5, 2017, 10:51pm UTC](https://discuss.elastic.co/t/nested-aggregation-count-seems-incorect/49687/5 "2017-07-05T22:51:42Z")

</div>


