# Terms Facet order by Count ... totally broken?

**URL:** https://discuss.elastic.co/t/terms-facet-order-by-count-totally-broken/12120
**Category:** Elasticsearch
**Created:** [May 24, 2013, 11:50pm UTC](https://discuss.elastic.co/t/terms-facet-order-by-count-totally-broken/12120 "2013-05-24T23:50:19Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![Nate\_Bauernfeind](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nate_bauernfeind/32/1993_2.png) [@Nate\_Bauernfeind](https://discuss.elastic.co/u/Nate_Bauernfeind)
#### Post date: [May 24, 2013, 11:50pm UTC](https://discuss.elastic.co/t/terms-facet-order-by-count-totally-broken/12120/1 "2013-05-24T23:50:19Z")

</div>

Simple Use Case:  
[https://gist.github.com/nbauernfeind/9e77615c7c2e57f6e8a5](https://gist.github.com/nbauernfeind/9e77615c7c2e57f6e8a5)

Basically, create 11 documents all of which contain 'A' and several other  
strings. A sample document:  
{  
"str": ["A", "B", "C", "D", "Q", "R"]  
}'

The do a terms facet ordered by count with a size = 1 (it should return the  
term 'a'):  
$ curl -XPOST "[http://localhost:9200/facet\_test/user/\_search?pretty=true](http://localhost:9200/facet_test/user/_search?pretty=true)"  
-d '

> {  
> "from" : 0,  
> "size" : 0,  
> "facets" : {  
> "str" : {  
> "terms" : {  
> "field" : "str",  
> "size" : 1,  
> "order" : "count"  
> }  
> }  
> }  
> }'  
> {  
> "took" : 1,  
> "timed\_out" : false,  
> "\_shards" : {  
> "total" : 5,  
> "successful" : 5,  
> "failed" : 0  
> },  
> "hits" : {  
> "total" : 11,  
> "max\_score" : 1.0,  
> "hits" :   
> },  
> "facets" : {  
> "str" : {  
> "\_type" : "terms",  
> "missing" : 1,  
> "total" : 33,  
> "other" : 28,  
> "terms" : [ {  
> "term" : "d",  
> "count" : 5  
> } ]  
> }  
> }  
> }

And see how even the returned value now returns a different count on this  
second query (with size = 2):  
$ curl -XPOST "[http://localhost:9200/facet\_test/user/\_search?pretty=true](http://localhost:9200/facet_test/user/_search?pretty=true)"  
-d '

> {  
> "from" : 0,  
> "size" : 0,  
> "facets" : {  
> "str" : {  
> "terms" : {  
> "field" : "str",  
> "size" : 2,  
> "order" : "count"  
> }  
> }  
> }  
> }'  
> {  
> "took" : 1,  
> "timed\_out" : false,  
> "\_shards" : {  
> "total" : 5,  
> "successful" : 5,  
> "failed" : 0  
> },  
> "hits" : {  
> "total" : 11,  
> "max\_score" : 1.0,  
> "hits" :   
> },  
> "facets" : {  
> "str" : {  
> "\_type" : "terms",  
> "missing" : 1,  
> "total" : 33,  
> "other" : 19,  
> "terms" : [ {  
> "term" : "d",  
> "count" : 7  
> }, {  
> "term" : "c",  
> "count" : 7  
> } ]  
> }  
> }  
> }

Is this a bug or do I have absolutely no idea how this should work?

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

### Author: ![mattweber](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mattweber/32/44940_2.png) [@mattweber](https://discuss.elastic.co/u/mattweber)
#### Post date: [May 25, 2013, 12:12am UTC](https://discuss.elastic.co/t/terms-facet-order-by-count-totally-broken/12120/2 "2013-05-25T00:12:49Z")

</div>

This is due to the low doc count and default 5 shards and the way counts  
are calculated across shards. See

> <https://github.com/elastic/elasticsearch/issues/1305>
>
> I'm working with nested documents and have noticed that my faceted search interf…ace is giving the wrong counts when I have more than one shard. To be more specific, I'm working with RDF triples (entity \> attribute \> value) and I'm nesting the attributes (called predicates in my example):
> 
> \`\`\`
> {
> "\_id" : "512a2c022f0b4e3daa341e6c8bcf6c2f",
> "url": "http://dbpedia.org/resource/Alan\_Shepard",
> "predicates": \[
> {
> "type": "type",
> "string\_value": \["thing", "person", "astronaut"\]
> }, {
> "type": "label",
> "string\_value": \["Alan Shepard"\]
> }, {
> "type": "time in space",
> "float\_value": \[216.950\]
> },
> ... lots more
> \]
> }
> \`\`\`
> 
> I've created a shell script (https://gist.github.com/1196986) that recreates the problem with a fresh index. The created data set has these totals:
> \- thing (30)
> \- creative work (20)
> \- video game (10)
> \- tv show (10)
> \- people (10)
> 
> With only \*\*one shard\*\* the following query gives the correct counts no matter what the size parameter is set to:
> 
> \`\`\`
> {
> "size": 0,
> "query": {
> "match\_all": {}
> },
> "facets": {
> "type\_counts": {
> "terms": {
> "field": "string\_value",
> "size": 5
> },
> "nested": "predicates",
> "facet\_filter": {
> "term": {
> "type": "type"
> }
> }
> }
> }
> }
> \`\`\`
> 
> However, with \*\*more than one shard\*\* the size parameter affects the accuracy of the counts. If it is equal to or greater than the number of terms returned by the facet query (5 in this case) then it works fine. However, the terms at the bottom of the list start to display low counts as you reduce the size parameter:
> 
> With "size" : 4
> \- thing (30)
> \- creative work (20)
> \- video game (10)
> \- \*\*tv show (9)\*\*
> 
> With "size" : 3
> \- thing (30)
> \- \*\*creative work (15)\*\*
> \- \*\*video game (9)\*\*
> 
> With "size" : 2
> \- thing (30)
> \- \*\*creative work (15)\*\*
> 
> So it looks like the sub-totals from some of the shards aren't being included for some reason. BTW I'm on ubuntu and the problem seems to affect all versions of ES I've tried (17.0, 17.1 and 17.6). Any ideas...?
> 
> P.S. absolutely loving ES - it's made my life a lot easier :)

On Friday, May 24, 2013, Nate Bauernfeind wrote:

> Simple Use Case:  
> [https://gist.github.com/nbauernfeind/9e77615c7c2e57f6e8a5](https://gist.github.com/nbauernfeind/9e77615c7c2e57f6e8a5)
> 
> Basically, create 11 documents all of which contain 'A' and several other  
> strings. A sample document:  
> {  
> "str": ["A", "B", "C", "D", "Q", "R"]  
> }'
> 
> The do a terms facet ordered by count with a size = 1 (it should return  
> the term 'a'):  
> $ curl -XPOST "[http://localhost:9200/facet\_test/user/\_search?pretty=true](http://localhost:9200/facet_test/user/_search?pretty=true)"  
> -d '
> 
> > {  
> > "from" : 0,  
> > "size" : 0,  
> > "facets" : {  
> > "str" : {  
> > "terms" : {  
> > "field" : "str",  
> > "size" : 1,  
> > "order" : "count"  
> > }  
> > }  
> > }  
> > }'  
> > {  
> > "took" : 1,  
> > "timed\_out" : false,  
> > "\_shards" : {  
> > "total" : 5,  
> > "successful" : 5,  
> > "failed" : 0  
> > },  
> > "hits" : {  
> > "total" : 11,  
> > "max\_score" : 1.0,  
> > "hits" :   
> > },  
> > "facets" : {  
> > "str" : {  
> > "\_type" : "terms",  
> > "missing" : 1,  
> > "total" : 33,  
> > "other" : 28,  
> > "terms" : [ {  
> > "term" : "d",  
> > "count" : 5  
> > } ]  
> > }  
> > }  
> > }
> 
> And see how even the returned value now returns a different count on this  
> second query (with size = 2):  
> $ curl -XPOST "[http://localhost:9200/facet\_test/user/\_search?pretty=true](http://localhost:9200/facet_test/user/_search?pretty=true)"  
> -d '
> 
> > {  
> > "from" : 0,  
> > "size" : 0,  
> > "facets" : {  
> > "str" : {  
> > "terms" : {  
> > "field" : "str",  
> > "size" : 2,  
> > "order" : "count"  
> > }  
> > }  
> > }  
> > }'  
> > {  
> > "took" : 1,  
> > "timed\_out" : false,  
> > "\_shards" : {  
> > "total" : 5,  
> > "successful" : 5,  
> > "failed" : 0  
> > },  
> > "hits" : {  
> > "total" : 11,  
> > "max\_score" : 1.0,  
> > "hits" :   
> > },  
> > "facets" : {  
> > "str" : {  
> > "\_type" : "terms",  
> > "missing" : 1,  
> > "total" : 33,  
> > "other" : 19,  
> > "terms" : [ {  
> > "term" : "d",  
> > "count" : 7  
> > }, {  
> > "term" : "c",  
> > "count" : 7  
> > } ]  
> > }  
> > }  
> > }
> 
> Is this a bug or do I have absolutely no idea how this should work?
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) \<javascript:\_e({},  
> 'cvml', 'elasticsearch%2Bunsubscribe@googlegroups.com');\>.  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
Thanks,  
Matt Weber

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [May 25, 2013, 12:34am UTC](https://discuss.elastic.co/t/terms-facet-order-by-count-totally-broken/12120/3 "2013-05-25T00:34:22Z")

</div>

Note also that A is a stopword (default standard analyzer for strings) so my guess is that it's not indexed.

--  
David 😉  
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 25 mai 2013 à 01:50, Nate Bauernfeind [nate.bauernfeind@gmail.com](mailto:nate.bauernfeind@gmail.com) a écrit :

Simple Use Case:  
[https://gist.github.com/nbauernfeind/9e77615c7c2e57f6e8a5](https://gist.github.com/nbauernfeind/9e77615c7c2e57f6e8a5)

Basically, create 11 documents all of which contain 'A' and several other strings. A sample document:  
{  
"str": ["A", "B", "C", "D", "Q", "R"]  
}'

The do a terms facet ordered by count with a size = 1 (it should return the term 'a'):  
$ curl -XPOST "[http://localhost:9200/facet\_test/user/\_search?pretty=true](http://localhost:9200/facet_test/user/_search?pretty=true)" -d '

> {  
> "from" : 0,  
> "size" : 0,  
> "facets" : {  
> "str" : {  
> "terms" : {  
> "field" : "str",  
> "size" : 1,  
> "order" : "count"  
> }  
> }  
> }  
> }'  
> {  
> "took" : 1,  
> "timed\_out" : false,  
> "\_shards" : {  
> "total" : 5,  
> "successful" : 5,  
> "failed" : 0  
> },  
> "hits" : {  
> "total" : 11,  
> "max\_score" : 1.0,  
> "hits" :   
> },  
> "facets" : {  
> "str" : {  
> "\_type" : "terms",  
> "missing" : 1,  
> "total" : 33,  
> "other" : 28,  
> "terms" : [ {  
> "term" : "d",  
> "count" : 5  
> } ]  
> }  
> }  
> }

And see how even the returned value now returns a different count on this second query (with size = 2):  
$ curl -XPOST "[http://localhost:9200/facet\_test/user/\_search?pretty=true](http://localhost:9200/facet_test/user/_search?pretty=true)" -d '

> {  
> "from" : 0,  
> "size" : 0,  
> "facets" : {  
> "str" : {  
> "terms" : {  
> "field" : "str",  
> "size" : 2,  
> "order" : "count"  
> }  
> }  
> }  
> }'  
> {  
> "took" : 1,  
> "timed\_out" : false,  
> "\_shards" : {  
> "total" : 5,  
> "successful" : 5,  
> "failed" : 0  
> },  
> "hits" : {  
> "total" : 11,  
> "max\_score" : 1.0,  
> "hits" :   
> },  
> "facets" : {  
> "str" : {  
> "\_type" : "terms",  
> "missing" : 1,  
> "total" : 33,  
> "other" : 19,  
> "terms" : [ {  
> "term" : "d",  
> "count" : 7  
> }, {  
> "term" : "c",  
> "count" : 7  
> } ]  
> }  
> }  
> }

## Is this a bug or do I have absolutely no idea how this should work?

You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

### Author: ![Nate\_Bauernfeind](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nate_bauernfeind/32/1993_2.png) [@Nate\_Bauernfeind](https://discuss.elastic.co/u/Nate_Bauernfeind)
#### Post date: [May 29, 2013, 6:36pm UTC](https://discuss.elastic.co/t/terms-facet-order-by-count-totally-broken/12120/4 "2013-05-29T18:36:05Z")

</div>

Thank you guys very much. Looks like I'll be going with an size = size \* 3  
approach and hope that gets me mostly accurate results.

Thanks,  
Nate

On Fri, May 24, 2013 at 7:34 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote:

> Note also that A is a stopword (default standard analyzer for strings) so  
> my guess is that it's not indexed.
> 
> --  
> David 😉  
> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
> 
> Le 25 mai 2013 à 01:50, Nate Bauernfeind [nate.bauernfeind@gmail.com](mailto:nate.bauernfeind@gmail.com) a  
> écrit :
> 
> Simple Use Case:  
> [https://gist.github.com/nbauernfeind/9e77615c7c2e57f6e8a5](https://gist.github.com/nbauernfeind/9e77615c7c2e57f6e8a5)
> 
> Basically, create 11 documents all of which contain 'A' and several other  
> strings. A sample document:  
> {  
> "str": ["A", "B", "C", "D", "Q", "R"]  
> }'
> 
> The do a terms facet ordered by count with a size = 1 (it should return  
> the term 'a'):  
> $ curl -XPOST "[http://localhost:9200/facet\_test/user/\_search?pretty=true](http://localhost:9200/facet_test/user/_search?pretty=true)"  
> -d '
> 
> > {  
> > "from" : 0,  
> > "size" : 0,  
> > "facets" : {  
> > "str" : {  
> > "terms" : {  
> > "field" : "str",  
> > "size" : 1,  
> > "order" : "count"  
> > }  
> > }  
> > }  
> > }'  
> > {  
> > "took" : 1,  
> > "timed\_out" : false,  
> > "\_shards" : {  
> > "total" : 5,  
> > "successful" : 5,  
> > "failed" : 0  
> > },  
> > "hits" : {  
> > "total" : 11,  
> > "max\_score" : 1.0,  
> > "hits" :   
> > },  
> > "facets" : {  
> > "str" : {  
> > "\_type" : "terms",  
> > "missing" : 1,  
> > "total" : 33,  
> > "other" : 28,  
> > "terms" : [ {  
> > "term" : "d",  
> > "count" : 5  
> > } ]  
> > }  
> > }  
> > }
> 
> And see how even the returned value now returns a different count on this  
> second query (with size = 2):  
> $ curl -XPOST "[http://localhost:9200/facet\_test/user/\_search?pretty=true](http://localhost:9200/facet_test/user/_search?pretty=true)"  
> -d '
> 
> > {  
> > "from" : 0,  
> > "size" : 0,  
> > "facets" : {  
> > "str" : {  
> > "terms" : {  
> > "field" : "str",  
> > "size" : 2,  
> > "order" : "count"  
> > }  
> > }  
> > }  
> > }'  
> > {  
> > "took" : 1,  
> > "timed\_out" : false,  
> > "\_shards" : {  
> > "total" : 5,  
> > "successful" : 5,  
> > "failed" : 0  
> > },  
> > "hits" : {  
> > "total" : 11,  
> > "max\_score" : 1.0,  
> > "hits" :   
> > },  
> > "facets" : {  
> > "str" : {  
> > "\_type" : "terms",  
> > "missing" : 1,  
> > "total" : 33,  
> > "other" : 19,  
> > "terms" : [ {  
> > "term" : "d",  
> > "count" : 7  
> > }, {  
> > "term" : "c",  
> > "count" : 7  
> > } ]  
> > }  
> > }  
> > }
> 
> Is this a bug or do I have absolutely no idea how this should work?
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> 
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> 
> --  
> You received this message because you are subscribed to a topic in the  
> Google Groups "elasticsearch" group.  
> To unsubscribe from this topic, visit  
> [https://groups.google.com/d/topic/elasticsearch/jNVnabFk1jY/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/jNVnabFk1jY/unsubscribe?hl=en-US)  
> .  
> To unsubscribe from this group and all its topics, send an email to  
> [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 2:34am UTC](https://discuss.elastic.co/t/terms-facet-order-by-count-totally-broken/12120/5 "2017-07-06T02:34:02Z")

</div>