# Bg\_counts in nested significant\_terms aggregation

**URL:** <https://discuss.elastic.co/t/bg-counts-in-nested-significant-terms-aggregation/21878>\
**Category:** Elasticsearch\
**Created:** [January 28, 2015, 2:07pm UTC](https://discuss.elastic.co/t/bg-counts-in-nested-significant-terms-aggregation/21878 "2015-01-28T14:07:12Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![elasticpete](https://avatars.discourse-cdn.com/v4/letter/e/90ced4/32.png) [@elasticpete](https://discuss.elastic.co/u/elasticpete)\
**Post date:** [January 28, 2015, 2:07pm UTC](https://discuss.elastic.co/t/bg-counts-in-nested-significant-terms-aggregation/21878/1 "2015-01-28T14:07:12Z")

</div>

When using a significant\_terms aggregation nested inside another aggregation, e.g. terms, I get different bg\_counts for the same significant term found across term buckets.  
Say e.g. the outer terms agg is on a field with US state codes ("CA", "FL, "NY", etc.) and the nested significant\_terms agg is on a field with the type of sport persons perform (e.g. "tennis", "golf", "skiing", etc.).  
I see the following types of results:

"aggregations": {  
"frequentTerms": {  
"buckets": [  
{  
"key": "NY",  
"doc\_count": 2027,  
"significantTerms": {  
"doc\_count": 2027,  
"buckets": [  
{  
"key": "sailing",  
"doc\_count": 80,  
"score": 0.029240945633836113,  
"bg\_count": 80  
},  
{  
"key": "golf",  
"doc\_count": 77,  
"score": 0.02907984745352633,  
"bg\_count": 77  
}  
]  
}  
}  
,  
{  
"key": "CA",  
"doc\_count": 100,  
"significantTerms": {  
"doc\_count": 100,  
"buckets": [  
{  
"key": "golf",  
"doc\_count": 42,  
"score": 0.02301730117174594,  
"bg\_count": 18  
},  
{  
"key": "tennis",  
"doc\_count": 42,  
"score": 0.012398130001513895,  
"bg\_count": 9  
}  
]  
}  
}  
]  
}  
}

I would expect that the bg\_count for "golf" would be identical for the two buckets (states). I have set the shard\_size to a very high number, and both min\_doc\_count and shard\_min\_doc\_count to 1, with no effect.

Any insights would be very appreciated.

Thanks, Petter.

---

<div class="post-metadata">

**Author:** ![tomlameche](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tomlameche/32/4832_2.png) [@tomlameche](https://discuss.elastic.co/u/tomlameche)\
**Post date:** [September 17, 2015, 3:21pm UTC](https://discuss.elastic.co/t/bg-counts-in-nested-significant-terms-aggregation/21878/2 "2015-09-17T15:21:33Z")

</div>

I see the same probleme, and very strange too, the \_superset\_freq is greater than doc\_count...

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [September 17, 2015, 4:39pm UTC](https://discuss.elastic.co/t/bg-counts-in-nested-significant-terms-aggregation/21878/3 "2015-09-17T16:39:33Z")

</div>

I expect the problem to be related to nested docs.  
They physically exist as separate docs in Lucene (from where we get some of our term frequency stats) but are accounted for differently in elasticsearch in things like top-level aggs where we like to pretend they don't exist.  
The Lucene APIs we rely on for fast access to frequencies are subject to inaccuracies due to things like deleted documents but it looks like nested docs are another source of potential inaccuracies

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:49pm UTC](https://discuss.elastic.co/t/bg-counts-in-nested-significant-terms-aggregation/21878/4 "2017-07-05T23:49:40Z")

</div>