# Wrong facet counts?

**URL:** <https://discuss.elastic.co/t/wrong-facet-counts/9284>\
**Category:** Elasticsearch\
**Created:** [October 9, 2012, 8:52am UTC](https://discuss.elastic.co/t/wrong-facet-counts/9284 "2012-10-09T08:52:25Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![guanyum](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guanyum/32/2658_2.png) [@guanyum](https://discuss.elastic.co/u/guanyum)\
**Post date:** [October 9, 2012, 8:52am UTC](https://discuss.elastic.co/t/wrong-facet-counts/9284/1 "2012-10-09T08:52:25Z")

</div>

hi all,

i indexed 4,230,593 docs, into a 3 es-node cluster, with 6 shards, 2  
index types.

i tried facet query, and the count number is less than my expect. here  
is my test:

query1 (match certain results)

```auto
curl -X POST "http://10.20.157.111:9200/sourcing/product/_search?
size=1&pretty=1" -d '
{
"query": {"query_string":{"query":"pub_cat_id:145"}},
"facets":{
 "pub_cat" : {"terms":{"field" : "pub_cat_id","size": 1}}}
}
'
and the counts for pub_cat_id:145 is 501,779, and it's the same as the
results i got from oracle.

query2 (match all query)

```

curl -X POST "[http://10.20.157.111:9200/sourcing/product/\_search](http://10.20.157.111:9200/sourcing/product/_search)?  
size=1&pretty=1" -d '  
{  
"query": {"match\_all":{}},  
"facets":{  
"pub\_cat" : {"terms":{"field" : "pub\_cat\_id","size": 1}}}  
}  
'  
and the counts for pub\_cat\_id:145 become less, and it's 500,337.

a bug here? or something i miss used? any comments or suggestions are  
much appreciated. thanks.

elasticsearch version is use is 0.19.9, and pub\_cat\_id is a single  
valued integer. i index doc like this,

XContentFactory.jsonBuilder().startObject().startObject(typeName)  
.field("pub\_cat\_id", aProductDo.getCategoryId())

--

---

<div class="post-metadata">

**Author:** ![ppearcy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ppearcy/32/980_2.png) [@ppearcy](https://discuss.elastic.co/u/ppearcy)\
**Post date:** [October 10, 2012, 3:19am UTC](https://discuss.elastic.co/t/wrong-facet-counts/9284/2 "2012-10-10T03:19:59Z")

</div>

Hi,  
Take a look at this discussion:  
[http://elasticsearch-users.115913.n3.nabble.com/Inconsistent-facet-count-td2977512.html](http://elasticsearch-users.115913.n3.nabble.com/Inconsistent-facet-count-td2977512.html)

Here is the key info:  
"This happens because of the way the distributed facet calculation works.  
It gets the top 5 from each shard, and then aggregates it. Because you have  
an even distribution of terms, it will not return exact matches. if you  
increase the size, you will get better results. One possible value can be  
the size times the number of shards, for example: 25."

Try increasing the facet count. It will help, but still isn't bullet proof.  
The only bullet proof (albeit crappy) solutions are:

- Have all your data in a single shard
- Have your count set to the number of distinct terms. This is overkill,  
though, and results are usual good enough following Shays guidance (size \*

# of shards)

Best Regards,  
Paul

On Tuesday, October 9, 2012 2:52:28 AM UTC-6, Brian Hu wrote:

> hi all,
> 
> i indexed 4,230,593 docs, into a 3 es-node cluster, with 6 shards, 2  
> index types.
> 
> i tried facet query, and the count number is less than my expect. here  
> is my test:
> 
> query1 (match certain results)
> 
> ```auto
> curl -X POST "http://10.20.157.111:9200/sourcing/product/_search? 
> size=1&pretty=1<http://10.20.157.111:9200/sourcing/product/_search?size=1&pretty=1>" 
> -d ' 
> { 
> "query": {"query_string":{"query":"pub_cat_id:145"}}, 
> "facets":{ 
> "pub_cat" : {"terms":{"field" : "pub_cat_id","size": 1}}} 
> } 
> ' 
> and the counts for pub_cat_id:145 is 501,779, and it's the same as the 
> results i got from oracle. 
> 
> query2 (match all query) 
> 
> ```
> 
> curl -X POST "[http://10.20.157.111:9200/sourcing/product/\_search](http://10.20.157.111:9200/sourcing/product/_search)?  
> size=1&pretty=1[http://10.20.157.111:9200/sourcing/product/\_search?size=1&pretty=1](http://10.20.157.111:9200/sourcing/product/_search?size=1&pretty=1)"  
> -d '  
> {  
> "query": {"match\_all":{}},  
> "facets":{  
> "pub\_cat" : {"terms":{"field" : "pub\_cat\_id","size": 1}}}  
> }  
> '  
> and the counts for pub\_cat\_id:145 become less, and it's 500,337.
> 
> a bug here? or something i miss used? any comments or suggestions are  
> much appreciated. thanks.
> 
> elasticsearch version is use is 0.19.9, and pub\_cat\_id is a single  
> valued integer. i index doc like this,
> 
> XContentFactory.jsonBuilder().startObject().startObject(typeName)  
> .field("pub\_cat\_id", aProductDo.getCategoryId())

--

---

<div class="post-metadata">

**Author:** ![ppearcy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ppearcy/32/980_2.png) [@ppearcy](https://discuss.elastic.co/u/ppearcy)\
**Post date:** [October 12, 2012, 6:43am UTC](https://discuss.elastic.co/t/wrong-facet-counts/9284/3 "2012-10-12T06:43:41Z")

</div>

Heya,  
Check out this thread:  
[http://elasticsearch-users.115913.n3.nabble.com/Inconsistent-facet-count-td2977512.html](http://elasticsearch-users.115913.n3.nabble.com/Inconsistent-facet-count-td2977512.html)

The key comment:  
This happens because of the way the distributed facet calculation works. It  
gets the top 5 from each shard, and then aggregates it. Because you have an  
even distribution of terms, it will not return exact matches. if you  
increase the size, you will get better results. One possible value can be  
the size times the number of shards, for example: 25.

Couple of reasonable, albeit not ideal, workarounds:

- Request a much higher count than you actually need
- If you need to guarantee exact counts, have a two pass query. Run the  
first query to get your initial list and feed those back in as part of the  
query to get the exact counts.

Best Regards,  
Paul

On Tuesday, October 9, 2012 2:52:28 AM UTC-6, Brian Hu wrote:

> hi all,
> 
> i indexed 4,230,593 docs, into a 3 es-node cluster, with 6 shards, 2  
> index types.
> 
> i tried facet query, and the count number is less than my expect. here  
> is my test:
> 
> query1 (match certain results)
> 
> ```auto
> curl -X POST "http://10.20.157.111:9200/sourcing/product/_search? 
> size=1&pretty=1<http://10.20.157.111:9200/sourcing/product/_search?size=1&pretty=1>" 
> -d ' 
> { 
> "query": {"query_string":{"query":"pub_cat_id:145"}}, 
> "facets":{ 
> "pub_cat" : {"terms":{"field" : "pub_cat_id","size": 1}}} 
> } 
> ' 
> and the counts for pub_cat_id:145 is 501,779, and it's the same as the 
> results i got from oracle. 
> 
> query2 (match all query) 
> 
> ```
> 
> curl -X POST "[http://10.20.157.111:9200/sourcing/product/\_search](http://10.20.157.111:9200/sourcing/product/_search)?  
> size=1&pretty=1[http://10.20.157.111:9200/sourcing/product/\_search?size=1&pretty=1](http://10.20.157.111:9200/sourcing/product/_search?size=1&pretty=1)"  
> -d '  
> {  
> "query": {"match\_all":{}},  
> "facets":{  
> "pub\_cat" : {"terms":{"field" : "pub\_cat\_id","size": 1}}}  
> }  
> '  
> and the counts for pub\_cat\_id:145 become less, and it's 500,337.
> 
> a bug here? or something i miss used? any comments or suggestions are  
> much appreciated. thanks.
> 
> elasticsearch version is use is 0.19.9, and pub\_cat\_id is a single  
> valued integer. i index doc like this,
> 
> XContentFactory.jsonBuilder().startObject().startObject(typeName)  
> .field("pub\_cat\_id", aProductDo.getCategoryId())

--

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 3:09am UTC](https://discuss.elastic.co/t/wrong-facet-counts/9284/4 "2017-07-06T03:09:08Z")

</div>