# Terms Facet other property behavior vs documentation question
**URL:** https://discuss.elastic.co/t/terms-facet-other-property-behavior-vs-documentation-question/10362
**Category:** Elasticsearch
**Created:** [January 16, 2013, 12:34am UTC](https://discuss.elastic.co/t/terms-facet-other-property-behavior-vs-documentation-question/10362 "2013-01-16T00:34:05Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@Kevin\_Fox\_2](https://discuss.elastic.co/u/Kevin_Fox_2)
#### Post date: [January 16, 2013, 12:34am UTC](https://discuss.elastic.co/t/terms-facet-other-property-behavior-vs-documentation-question/10362/1 "2013-01-16T00:34:05Z")
The documentation at:
[http://www.elasticsearch.org/guide/reference/api/search/facets/](http://www.elasticsearch.org/guide/reference/api/search/facets/)
States:
"the number of facet values not included in the returned facets (other), "
In testing things, the number of facet entries returned in my query is
limited to 10 and other is saying something on the order of 200,000. If I
set the limit very high, there are only 34 facets returned with other = 0.
So, other's behavior does not seem to match up with the documentation? Is
the other field actually the number of documents that should match the
facet but aren't in one of the 10 returned facets?
Is there a way to get the count of all the unique facet rows that match
without getting the values so I can safely give the user the option to see
them all only if there aren't too many?
Thanks,
Kevin
--
---
### Author:  [@Ivan](https://discuss.elastic.co/u/Ivan)
#### Post date: [January 16, 2013, 3:42pm UTC](https://discuss.elastic.co/t/terms-facet-other-property-behavior-vs-documentation-question/10362/2 "2013-01-16T15:42:04Z")
Your assumption is correct, the other field is not how many other facets
are remaining, but the total number of documents.
There is no way to get all the unique facets. I believe there might be an
issue on Github. Jorg wrote a termlist plugin, which will get all the
unique terms for a field. If your facet is a term facet, then this plugin
would be equivalent.
Getting the entire term list can be slow, depending on your data. If you
have only 34 facets and you know this number will not change much, you can
try increasing the number of facets returned. It all depends on the size of
your data.
Cheers,
Ivan
On Tue, Jan 15, 2013 at 4:34 PM, Kevin Fox [kfox1111@gmail.com](mailto:kfox1111@gmail.com) wrote:
> Is there a way to get the count of all the unique facet rows that match
> without getting the values so I can safely give the user the option to see
> them all only if there aren't too many?
--
---
### Author:  [@Kevin\_Fox\_2](https://discuss.elastic.co/u/Kevin_Fox_2)
#### Post date: [January 16, 2013, 8:47pm UTC](https://discuss.elastic.co/t/terms-facet-other-property-behavior-vs-documentation-question/10362/3 "2013-01-16T20:47:20Z")
Bummer. Well, what I'm trying to do is display the top 10 facets in the
term. If there are more then 10, display a more button which pops up a
dialog that reissues the query requesting and displaying all of the facets.
This works very well except when the facet count for the term is huge. like
> 1. If it is that large, its probably not going to be useful to the
> user anyway, so I want to just ignore the whole term if its that large. If
> I could get the top 10 hits plus the number of remaining ones, the ui work
> would be simple. Plus it would be low overhead over the network. I'm
> thinking, since the sort by count on the term happens already, the whole
> list of facets must be loaded into memory and is a complete list. I should
> be able to just add an "extra" field or something that simply returns the
> number of facets in that list minus the ones returned. I think the code
> would be a simple patch to
> src/main/java/org/elasticsearch/search/facet/terms/strings/InternalStringTermsFacet.java.
> I this a reasonable thing to do?
Thanks,
Kevin
On Wednesday, January 16, 2013 7:42:04 AM UTC-8, Ivan Brusic wrote:
> Your assumption is correct, the other field is not how many other facets
> are remaining, but the total number of documents.
>
> There is no way to get all the unique facets. I believe there might be an
> issue on Github. Jorg wrote a termlist plugin, which will get all the
> unique terms for a field. If your facet is a term facet, then this plugin
> would be equivalent.
>
> Getting the entire term list can be slow, depending on your data. If you
> have only 34 facets and you know this number will not change much, you can
> try increasing the number of facets returned. It all depends on the size of
> your data.
>
> Cheers,
>
> Ivan
>
> On Tue, Jan 15, 2013 at 4:34 PM, Kevin Fox \<[kfox...@gmail.com](mailto:kfox...@gmail.com)\
>
> > wrote:
>
> > Is there a way to get the count of all the unique facet rows that match
> > without getting the values so I can safely give the user the option to see
> > them all only if there aren't too many?
--
---
### Author:  [@Ivan](https://discuss.elastic.co/u/Ivan)
#### Post date: [January 16, 2013, 9:13pm UTC](https://discuss.elastic.co/t/terms-facet-other-property-behavior-vs-documentation-question/10362/4 "2013-01-16T21:13:51Z")
BTW, my previous response was incorrect in that you can ask for all the
terms back. The setting is "all\_terms" : true. Since my use case is similar
to yours, where getting all terms is inefficient, I have never done so.
On Wed, Jan 16, 2013 at 12:47 PM, Kevin Fox [kfox1111@gmail.com](mailto:kfox1111@gmail.com) wrote:
> Bummer. Well, what I'm trying to do is display the top 10 facets in the
> term. If there are more then 10, display a more button which pops up a
> dialog that reissues the query requesting and displaying all of the facets.
> This works very well except when the facet count for the term is huge. like
>
> > 1. If it is that large, its probably not going to be useful to the
> > user anyway, so I want to just ignore the whole term if its that large. If
> > I could get the top 10 hits plus the number of remaining ones, the ui work
> > would be simple. Plus it would be low overhead over the network. I'm
> > thinking, since the sort by count on the term happens already, the whole
> > list of facets must be loaded into memory and is a complete list. I should
> > be able to just add an "extra" field or something that simply returns the
> > number of facets in that list minus the ones returned. I think the code
> > would be a simple patch to
> > src/main/java/org/elasticsearch/search/facet/terms/strings/InternalStringTermsFacet.java.
> > I this a reasonable thing to do?
>
> Thanks,
> Kevin
>
> On Wednesday, January 16, 2013 7:42:04 AM UTC-8, Ivan Brusic wrote:
>
> > Your assumption is correct, the other field is not how many other facets
> > are remaining, but the total number of documents.
> >
> > There is no way to get all the unique facets. I believe there might be an
> > issue on Github. Jorg wrote a termlist plugin, which will get all the
> > unique terms for a field. If your facet is a term facet, then this plugin
> > would be equivalent.
> >
> > Getting the entire term list can be slow, depending on your data. If you
> > have only 34 facets and you know this number will not change much, you can
> > try increasing the number of facets returned. It all depends on the size of
> > your data.
> >
> > Cheers,
> >
> > Ivan
> >
> > On Tue, Jan 15, 2013 at 4:34 PM, Kevin Fox [kfox...@gmail.com](mailto:kfox...@gmail.com) wrote:
> >
> > > Is there a way to get the count of all the unique facet rows that match
> > > without getting the values so I can safely give the user the option to see
> > > them all only if there aren't too many?
> >
> > --
--
---
### Author:  [@Kevin\_Fox\_2](https://discuss.elastic.co/u/Kevin_Fox_2)
#### Post date: [January 16, 2013, 10:46pm UTC](https://discuss.elastic.co/t/terms-facet-other-property-behavior-vs-documentation-question/10362/5 "2013-01-16T22:46:00Z")
So, the all\_terms field says it returns all of the facets with counts = 0
even if there is nothing that matches that facet. That means, if there are
a lot of them, it sends them all across the wire. What I'm trying to do is
get the count of all facets minus the elements returned, so I can determine
if I want to requery with something like all\_terms set to true or if it
would be too expensive to do so. Any idea where in the code I could add
that?
Thanks,
Kevin
On Wednesday, January 16, 2013 1:13:51 PM UTC-8, Ivan Brusic wrote:
> BTW, my previous response was incorrect in that you can ask for all the
> terms back. The setting is "all\_terms" : true. Since my use case is similar
> to yours, where getting all terms is inefficient, I have never done so.
>
> On Wed, Jan 16, 2013 at 12:47 PM, Kevin Fox \<[kfox...@gmail.com](mailto:kfox...@gmail.com)\
>
> > wrote:
>
> > Bummer. Well, what I'm trying to do is display the top 10 facets in the
> > term. If there are more then 10, display a more button which pops up a
> > dialog that reissues the query requesting and displaying all of the facets.
> > This works very well except when the facet count for the term is huge. like
> >
> > > 1. If it is that large, its probably not going to be useful to the
> > > user anyway, so I want to just ignore the whole term if its that large. If
> > > I could get the top 10 hits plus the number of remaining ones, the ui work
> > > would be simple. Plus it would be low overhead over the network. I'm
> > > thinking, since the sort by count on the term happens already, the whole
> > > list of facets must be loaded into memory and is a complete list. I should
> > > be able to just add an "extra" field or something that simply returns the
> > > number of facets in that list minus the ones returned. I think the code
> > > would be a simple patch to
> > > src/main/java/org/elasticsearch/search/facet/terms/strings/InternalStringTermsFacet.java.
> > > I this a reasonable thing to do?
> >
> > Thanks,
> > Kevin
> >
> > On Wednesday, January 16, 2013 7:42:04 AM UTC-8, Ivan Brusic wrote:
> >
> > > Your assumption is correct, the other field is not how many other facets
> > > are remaining, but the total number of documents.
> > >
> > > There is no way to get all the unique facets. I believe there might be
> > > an issue on Github. Jorg wrote a termlist plugin, which will get all the
> > > unique terms for a field. If your facet is a term facet, then this plugin
> > > would be equivalent.
> > >
> > > Getting the entire term list can be slow, depending on your data. If you
> > > have only 34 facets and you know this number will not change much, you can
> > > try increasing the number of facets returned. It all depends on the size of
> > > your data.
> > >
> > > Cheers,
> > >
> > > Ivan
> > >
> > > On Tue, Jan 15, 2013 at 4:34 PM, Kevin Fox [kfox...@gmail.com](mailto:kfox...@gmail.com) wrote:
> > >
> > > > Is there a way to get the count of all the unique facet rows that match
> > > > without getting the values so I can safely give the user the option to see
> > > > them all only if there aren't too many?
> > >
> > > --
--
---
### Author:  [@Ivan](https://discuss.elastic.co/u/Ivan)
#### Post date: [January 16, 2013, 11:25pm UTC](https://discuss.elastic.co/t/terms-facet-other-property-behavior-vs-documentation-question/10362/6 "2013-01-16T23:25:32Z")
The terms facets works off the termlist found in the Lucene index. There
can never be a terms facet with a zero count. Facets never return the
elements, just the term and the count.
If you just want a list of terms, take a long at Jorg's plugin:
> **[GitHub - jprante/elasticsearch-index-termlist: Elasticsearch Index Termlist](https://github.com/jprante/elasticsearch-index-termlist)**
>
> Elasticsearch Index Termlist. Contribute to jprante/elasticsearch-index-termlist development by creating an account on GitHub.
The Lucene code is found here:
[https://github.com/jprante/elasticsearch-index-termlist/blob/master/src/main/java/org/elasticsearch/action/termlist/TransportTermlistAction.java#L126](https://github.com/jprante/elasticsearch-index-termlist/blob/master/src/main/java/org/elasticsearch/action/termlist/TransportTermlistAction.java#L126)
The plugin aggregates the termlists from each shard.
--
Ivan
On Wed, Jan 16, 2013 at 2:46 PM, Kevin Fox [kfox1111@gmail.com](mailto:kfox1111@gmail.com) wrote:
> So, the all\_terms field says it returns all of the facets with counts = 0
> even if there is nothing that matches that facet. That means, if there are
> a lot of them, it sends them all across the wire. What I'm trying to do is
> get the count of all facets minus the elements returned, so I can determine
> if I want to requery with something like all\_terms set to true or if it
> would be too expensive to do so. Any idea where in the code I could add
> that?
>
> Thanks,
> Kevin
--
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [January 17, 2013, 9:06am UTC](https://discuss.elastic.co/t/terms-facet-other-property-behavior-vs-documentation-question/10362/7 "2013-01-17T09:06:13Z")
Hey Kevin,
I probably did not understand your use case. Why the standard Term Facet did not
answer to your need?
There is an "other" field which is part of the answer:
"facets" : {
"f1" : {
"\_type" : "terms",
"missing" : 0,
"total" : 3,
"other" : 0,
"terms" : [ {
"term" : "b",
"count" : 2
}, {
"term" : "c",
"count" : 1
} ]
}
}
This field means that there are other terms than the TOP 10 (
>
>
> As discussed in this thread, here is a feature request for terms facet : http://…elasticsearch-users.115913.n3.nabble.com/Terms-facet-getting-all-the-results-tp3051086p3055360.html
>
> I would like to get within the terms facet result the count of all other terms that have not been returned (due to size limit).
>
> Here is a test case :
>
> \`\`\` bash
> \# Dropping index and Creating articles
> curl -X DELETE "http://localhost:9200/articles"
> curl -X POST "http://localhost:9200/articles/article" -d '{"tags" : \["one", "two", "three", "four", "five", "six", "seven"\]}'
> curl -X POST "http://localhost:9200/articles/article" -d '{"tags" : \["two", "three", "four", "five", "six", "seven"\]}'
> curl -X POST "http://localhost:9200/articles/article" -d '{"tags" : \["three", "four", "five", "six", "seven"\]}'
> curl -X POST "http://localhost:9200/articles/article" -d '{"tags" : \["four", "five", "six", "seven"\]}'
> curl -X POST "http://localhost:9200/articles/article" -d '{"tags" : \["five", "six", "seven"\]}'
> curl -X POST "http://localhost:9200/articles/article" -d '{"tags" : \["six", "seven"\]}'
> curl -X POST "http://localhost:9200/articles/article" -d '{"tags" : \["seven"\]}'
> \`\`\`
>
> \`\`\` bash
> \# Classic search with terms facet
> curl -X POST "http://localhost:9200/articles/\_search?pretty=true" -d '
> {
> "query" : { "match\_all" : { } },
> "facets" : {
> "f\_tags" : { "terms" : {"field" : "tags"} }
> }
> }
> '
> \`\`\`
>
> \`\`\` javascript
> {
> "took" : 0,
> "timed\_out" : false,
> "\_shards" : {
> "total" : 5,
> "successful" : 5,
> "failed" : 0
> },
> "hits" : {
> "total" : 7,
> "max\_score" : 1.0,
> "hits" : \[ {
> "\_index" : "articles",
> "\_type" : "article",
> "\_id" : "gkgAyLN6QWuxSyBLxxknIw",
> "\_score" : 1.0, "\_source" : {"tags":\["two","three","four","five","six","seven"\]}
> }, {
> "\_index" : "articles",
> "\_type" : "article",
> "\_id" : "swVlq23SS9aoC8uHNB\_LRA",
> "\_score" : 1.0, "\_source" : {"tags":\["one","two","three","four","five","six","seven"\]}
> }, {
> "\_index" : "articles",
> "\_type" : "article",
> "\_id" : "oColEdY-SUmMPWvBWBEneQ",
> "\_score" : 1.0, "\_source" : {"tags":\["six","seven"\]}
> }, {
> "\_index" : "articles",
> "\_type" : "article",
> "\_id" : "hk-9osIlRViESqEF7WPj\_w",
> "\_score" : 1.0, "\_source" : {"tags":\["three","four","five","six","seven"\]}
> }, {
> "\_index" : "articles",
> "\_type" : "article",
> "\_id" : "Ki9KA036TI6n7f6vD2gGWA",
> "\_score" : 1.0, "\_source" : {"tags":\["four","five","six","seven"\]}
> }, {
> "\_index" : "articles",
> "\_type" : "article",
> "\_id" : "\_bFnruq4TdyJQjHH5bTMrg",
> "\_score" : 1.0, "\_source" : {"tags":\["five","six","seven"\]}
> }, {
> "\_index" : "articles",
> "\_type" : "article",
> "\_id" : "k1FzqoBwQ82PYNa1jgE-Tg",
> "\_score" : 1.0, "\_source" : {"tags":\["seven"\]}
> } \]
> },
> "facets" : {
> "f\_tags" : {
> "\_type" : "terms",
> "missing" : 0,
> "terms" : \[ {
> "term" : "seven",
> "count" : 7
> }, {
> "term" : "six",
> "count" : 6
> }, {
> "term" : "five",
> "count" : 5
> }, {
> "term" : "four",
> "count" : 4
> }, {
> "term" : "three",
> "count" : 3
> } \]
> }
> }
> }
> \`\`\`
>
> I would like to add an option to the term facet :
>
> \`\`\` bash
> \# New option "other\_terms" for terms facet
> curl -X POST "http://localhost:9200/articles/\_search?pretty=true" -d '
> {
> "query" : { "match\_all" : { } },
> "facets" : {
> "f\_tags" : { "terms" : {
> "field" : "tags",
> "other\_terms" : true
> } }
> }
> }
> '
> \`\`\`
>
> Which should return :
>
> \`\`\` javascript
> {
> "took" : 0,
> "timed\_out" : false,
> "\_shards" : {
> "total" : 5,
> "successful" : 5,
> "failed" : 0
> },
> "hits" : {
> "total" : 7,
> "max\_score" : 1.0,
> "hits" : \[ {
> "\_index" : "articles",
> "\_type" : "article",
> "\_id" : "gkgAyLN6QWuxSyBLxxknIw",
> "\_score" : 1.0, "\_source" : {"tags":\["two","three","four","five","six","seven"\]}
> }, {
> "\_index" : "articles",
> "\_type" : "article",
> "\_id" : "swVlq23SS9aoC8uHNB\_LRA",
> "\_score" : 1.0, "\_source" : {"tags":\["one","two","three","four","five","six","seven"\]}
> }, {
> "\_index" : "articles",
> "\_type" : "article",
> "\_id" : "oColEdY-SUmMPWvBWBEneQ",
> "\_score" : 1.0, "\_source" : {"tags":\["six","seven"\]}
> }, {
> "\_index" : "articles",
> "\_type" : "article",
> "\_id" : "hk-9osIlRViESqEF7WPj\_w",
> "\_score" : 1.0, "\_source" : {"tags":\["three","four","five","six","seven"\]}
> }, {
> "\_index" : "articles",
> "\_type" : "article",
> "\_id" : "Ki9KA036TI6n7f6vD2gGWA",
> "\_score" : 1.0, "\_source" : {"tags":\["four","five","six","seven"\]}
> }, {
> "\_index" : "articles",
> "\_type" : "article",
> "\_id" : "\_bFnruq4TdyJQjHH5bTMrg",
> "\_score" : 1.0, "\_source" : {"tags":\["five","six","seven"\]}
> }, {
> "\_index" : "articles",
> "\_type" : "article",
> "\_id" : "k1FzqoBwQ82PYNa1jgE-Tg",
> "\_score" : 1.0, "\_source" : {"tags":\["seven"\]}
> } \]
> },
> "facets" : {
> "f\_tags" : {
> "\_type" : "terms",
> "missing" : 0,
> "terms" : \[ {
> "term" : "seven",
> "count" : 7
> }, {
> "term" : "six",
> "count" : 6
> }, {
> "term" : "five",
> "count" : 5
> }, {
> "term" : "four",
> "count" : 4
> }, {
> "term" : "three",
> "count" : 3
> }, {
> "term" : "\_other",
> "count" : 3
> } \]
> }
> }
> }
> \`\`\`
>
> As you can see, a new \_other term field is returned with a count of 3 (2 terms "two" + 1 term "one").
>
> Many thanks
> Cheers
> David.
[https://github.com/elasticsearch/elasticsearch/issues/1029](https://github.com/elasticsearch/elasticsearch/issues/1029) )
Isn't it what you are after?
David.
Le 17 janvier 2013 à 00:25, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) a écrit :
> The terms facets works off the termlist found in the Lucene index. There can
> never be a terms facet with a zero count. Facets never return the elements,
> just the term and the count.
>
> If you just want a list of terms, take a long at Jorg's plugin:
> [GitHub - jprante/elasticsearch-index-termlist: Elasticsearch Index Termlist](https://github.com/jprante/elasticsearch-index-termlist)
> [https://github.com/jprante/elasticsearch-index-termlist](https://github.com/jprante/elasticsearch-index-termlist)
>
> The Lucene code is found here:
>
> [https://github.com/jprante/elasticsearch-index-termlist/blob/master/src/main/java/org/elasticsearch/action/termlist/TransportTermlistAction.java#L126](https://github.com/jprante/elasticsearch-index-termlist/blob/master/src/main/java/org/elasticsearch/action/termlist/TransportTermlistAction.java#L126)
> [https://github.com/jprante/elasticsearch-index-termlist/blob/master/src/main/java/org/elasticsearch/action/termlist/TransportTermlistAction.java#L126](https://github.com/jprante/elasticsearch-index-termlist/blob/master/src/main/java/org/elasticsearch/action/termlist/TransportTermlistAction.java#L126)
>
> The plugin aggregates the termlists from each shard.
>
> --
> Ivan
>
> On Wed, Jan 16, 2013 at 2:46 PM, Kevin Fox \<[kfox1111@gmail.com](mailto:kfox1111@gmail.com)
> [mailto:kfox1111@gmail.com](mailto:kfox1111@gmail.com) \> wrote:
>
> > > So, the all\_terms field says it returns all of the facets with
> > > counts = 0 even if there is nothing that matches that facet. That
> > > means, if there are a lot of them, it sends them all across the wire.
> > > What I'm trying to do is get the count of all facets minus the elements
> > > returned, so I can determine if I want to requery with something like
> > > all\_terms set to true or if it would be too expensive to do so. Any
> > > idea where in the code I could add that?
> >
> > Thanks,
> > Kevin
> >
> > >
>
> --
--
David Pilato
[http://www.scrutmydocs.org/](http://www.scrutmydocs.org/)
[http://dev.david.pilato.fr/](http://dev.david.pilato.fr/)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
--
---
### Author:  [@Kevin\_Fox\_2](https://discuss.elastic.co/u/Kevin_Fox_2)
#### Post date: [January 17, 2013, 4:06pm UTC](https://discuss.elastic.co/t/terms-facet-other-property-behavior-vs-documentation-question/10362/8 "2013-01-17T16:06:07Z")
Hi David,
No, unfortunately, I'm after something slightly different. My terminology
hasn't been so good, so let me try to state it again.
For a given term facet request, you limit the number of results with
"size". (default 10 terms)
Returned is
{
"terms" \<- up to "size" number of "name":"count" pairs
"total": \<- the number of _documents_ matched by the facet
"missing": \<- the number of _documents_ not having the property the facet
wants
"other": \<- "total" minus the number of _documents_ listed in "terms".
}
I'm interested in _terms_, not _documents_ for my particular problem. In
particular, you can get the number of terms matched: terms.length, but you
can't get the number of terms not returned. That is the value I'm
interested in.
"The number of other terms not returned". The only way currently is to set
size = HUGENUMBER and request them all, then count them. I'm not interested
in their names and/or counts, or sending them all over the network, simply
how many more of them there are during the initial query.
Does that make sense?
I'm looking through the code trying to figure out where to add that
feature. I'm most of the way there I think but haven't found quite the
right place where all the terms are assembled yet.
Thanks,
Kevin
On Thursday, January 17, 2013 1:06:13 AM UTC-8, David Pilato wrote:
> Hey Kevin,
>
> I probably did not understand your use case. Why the standard Term Facet
> did not answer to your need?
> There is an "other" field which is part of the answer:
>
> "facets" : {
> "f1" : {
> "\_type" : "terms",
> "missing" : 0,
> "total" : 3,
> "other" : 0,
> "terms" : [ {
> "term" : "b",
> "count" : 2
> }, {
> "term" : "c",
> "count" : 1
> } ]
> }
> }
> This field means that there are other terms than the TOP 10 (
> [Add 'other\_terms' option for terms facet · Issue #1029 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/1029))
> Isn't it what you are after?
>
> David.
>
> Le 17 janvier 2013 à 00:25, Ivan Brusic \<[iv...@brusic.com](mailto:iv...@brusic.com) \\>
> a écrit :
>
> The terms facets works off the termlist found in the Lucene index. There
> can never be a terms facet with a zero count. Facets never return the
> elements, just the term and the count.
>
> If you just want a list of terms, take a long at Jorg's plugin:
> [GitHub - jprante/elasticsearch-index-termlist: Elasticsearch Index Termlist](https://github.com/jprante/elasticsearch-index-termlist)
>
> The Lucene code is found here:
>
> [https://github.com/jprante/elasticsearch-index-termlist/blob/master/src/main/java/org/elasticsearch/action/termlist/TransportTermlistAction.java#L126](https://github.com/jprante/elasticsearch-index-termlist/blob/master/src/main/java/org/elasticsearch/action/termlist/TransportTermlistAction.java#L126)
>
> The plugin aggregates the termlists from each shard.
>
> --
> Ivan
>
> On Wed, Jan 16, 2013 at 2:46 PM, Kevin Fox \<[kfox...@gmail.com](mailto:kfox...@gmail.com)\
>
> > wrote:
>
> So, the all\_terms field says it returns all of the facets with counts = 0
> even if there is nothing that matches that facet. That means, if there are
> a lot of them, it sends them all across the wire. What I'm trying to do is
> get the count of all facets minus the elements returned, so I can determine
> if I want to requery with something like all\_terms set to true or if it
> would be too expensive to do so. Any idea where in the code I could add
> that?
>
> Thanks,
> Kevin
>
> --
>
> --
> David Pilato
> [http://www.scrutmydocs.org/](http://www.scrutmydocs.org/)
> [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/)
> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
--
---
### Author:  [@Kevin\_Fox\_2](https://discuss.elastic.co/u/Kevin_Fox_2)
#### Post date: [January 18, 2013, 5:14pm UTC](https://discuss.elastic.co/t/terms-facet-other-property-behavior-vs-documentation-question/10362/9 "2013-01-18T17:14:56Z")
Ok, here is a patch that adds an "extra" field that returns the number of
additional terms in the facet when the terms are strings. Its only been
lightly tested and doesn't work on the other data types yet but it is a
start.
What do you think?
Thanks,
Kevin
On Thursday, January 17, 2013 8:06:07 AM UTC-8, Kevin Fox wrote:
> Hi David,
>
> No, unfortunately, I'm after something slightly different. My terminology
> hasn't been so good, so let me try to state it again.
>
> For a given term facet request, you limit the number of results with
> "size". (default 10 terms)
>
> Returned is
> {
> "terms" \<- up to "size" number of "name":"count" pairs
> "total": \<- the number of _documents_ matched by the facet
> "missing": \<- the number of _documents_ not having the property the facet
> wants
> "other": \<- "total" minus the number of _documents_ listed in "terms".
> }
>
> I'm interested in _terms_, not _documents_ for my particular problem. In
> particular, you can get the number of terms matched: terms.length, but you
> can't get the number of terms not returned. That is the value I'm
> interested in.
> "The number of other terms not returned". The only way currently is to set
> size = HUGENUMBER and request them all, then count them. I'm not interested
> in their names and/or counts, or sending them all over the network, simply
> how many more of them there are during the initial query.
>
> Does that make sense?
>
> I'm looking through the code trying to figure out where to add that
> feature. I'm most of the way there I think but haven't found quite the
> right place where all the terms are assembled yet.
>
> Thanks,
> Kevin
>
> On Thursday, January 17, 2013 1:06:13 AM UTC-8, David Pilato wrote:
>
> > Hey Kevin,
> >
> > I probably did not understand your use case. Why the standard Term Facet
> > did not answer to your need?
> > There is an "other" field which is part of the answer:
> >
> > "facets" : {
> > "f1" : {
> > "\_type" : "terms",
> > "missing" : 0,
> > "total" : 3,
> > "other" : 0,
> > "terms" : [ {
> > "term" : "b",
> > "count" : 2
> > }, {
> > "term" : "c",
> > "count" : 1
> > } ]
> > }
> > }
> > This field means that there are other terms than the TOP 10 (
> > [Add 'other\_terms' option for terms facet · Issue #1029 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/1029))
> > Isn't it what you are after?
> >
> > David.
> >
> > Le 17 janvier 2013 à 00:25, Ivan Brusic [iv...@brusic.com](mailto:iv...@brusic.com) a écrit :
> >
> > The terms facets works off the termlist found in the Lucene index.
> > There can never be a terms facet with a zero count. Facets never return the
> > elements, just the term and the count.
> >
> > If you just want a list of terms, take a long at Jorg's plugin:
> > [GitHub - jprante/elasticsearch-index-termlist: Elasticsearch Index Termlist](https://github.com/jprante/elasticsearch-index-termlist)
> >
> > The Lucene code is found here:
> >
> > [https://github.com/jprante/elasticsearch-index-termlist/blob/master/src/main/java/org/elasticsearch/action/termlist/TransportTermlistAction.java#L126](https://github.com/jprante/elasticsearch-index-termlist/blob/master/src/main/java/org/elasticsearch/action/termlist/TransportTermlistAction.java#L126)
> >
> > The plugin aggregates the termlists from each shard.
> >
> > --
> > Ivan
> >
> > On Wed, Jan 16, 2013 at 2:46 PM, Kevin Fox [kfox...@gmail.com](mailto:kfox...@gmail.com) wrote:
> >
> > So, the all\_terms field says it returns all of the facets with counts =
> > 0 even if there is nothing that matches that facet. That means, if there
> > are a lot of them, it sends them all across the wire. What I'm trying to do
> > is get the count of all facets minus the elements returned, so I can
> > determine if I want to requery with something like all\_terms set to true or
> > if it would be too expensive to do so. Any idea where in the code I could
> > add that?
> >
> > Thanks,
> > Kevin
> >
> > --
> >
> > --
> > David Pilato
> > [http://www.scrutmydocs.org/](http://www.scrutmydocs.org/)
> > [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/)
> > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
--
---
### Author:  [@Ivan](https://discuss.elastic.co/u/Ivan)
#### Post date: [January 22, 2013, 10:31pm UTC](https://discuss.elastic.co/t/terms-facet-other-property-behavior-vs-documentation-question/10362/10 "2013-01-22T22:31:46Z")
Replying back to the group since there are others that know the internals
far better than I do.
The reduce() function is where the aggregation is happening. Depending on
the distribution of your terms, your method could return the correct number
on occasion. Hopefully someone else will respond.
--
Ivan
On Mon, Jan 21, 2013 at 8:34 AM, Kevin Fox [kfox1111@gmail.com](mailto:kfox1111@gmail.com) wrote:
> I'm not sure. The code is a bit hard to follow. When I instrumented it, I
> got #ofshards instances of InternalStringTermsFacet in
> TermsStringOrdinalsFacetCollector, one of which had the right number. The
> others were way smaller. I was thinking things probably got aggregated to
> that one. The max then just finds the entry. I could be totally wrong
> though. Do you know if/where such an aggregation might happen?
>
> Thanks,
> Kevin
> On Jan 18, 2013 9:39 AM, "Ivan Brusic" [ivan@brusic.com](mailto:ivan@brusic.com) wrote:
>
> > I haven't stepped through the code, but isn't the handling of extra
> > during the reduce phase a bit optimistic?
> >
> > extra = Math.max(extra, mFacet.extraCount());
> >
> > If each shard returns n unique extra facets, then the extra count would
> > be n\*number\_of\_shards, not n. Without access to the actual facet terms,
> > there is no way to detect duplication (or lack of). That is why
> > the aggregated facets are held in a hash map. Perhaps I'm wrong.
> >
> > --
> > Ivan
> >
> > On Fri, Jan 18, 2013 at 9:14 AM, Kevin Fox [kfox1111@gmail.com](mailto:kfox1111@gmail.com) wrote:
> >
> > > Ok, here is a patch that adds an "extra" field that returns the number
> > > of additional terms in the facet when the terms are strings. Its only been
> > > lightly tested and doesn't work on the other data types yet but it is a
> > > start.
> > >
> > > What do you think?
> > >
> > > Thanks,
> > > Kevin
> > >
> > > On Thursday, January 17, 2013 8:06:07 AM UTC-8, Kevin Fox wrote:
> > >
> > > > Hi David,
> > > >
> > > > No, unfortunately, I'm after something slightly different. My
> > > > terminology hasn't been so good, so let me try to state it again.
> > > >
> > > > For a given term facet request, you limit the number of results with
> > > > "size". (default 10 terms)
> > > >
> > > > Returned is
> > > > {
> > > > "terms" \<- up to "size" number of "name":"count" pairs
> > > > "total": \<- the number of _documents_ matched by the facet
> > > > "missing": \<- the number of _documents_ not having the property the
> > > > facet wants
> > > > "other": \<- "total" minus the number of _documents_ listed in "terms".
> > > > }
> > > >
> > > > I'm interested in _terms_, not _documents_ for my particular problem.
> > > > In particular, you can get the number of terms matched: terms.length, but
> > > > you can't get the number of terms not returned. That is the value I'm
> > > > interested in.
> > > > "The number of other terms not returned". The only way currently is to
> > > > set size = HUGENUMBER and request them all, then count them. I'm not
> > > > interested in their names and/or counts, or sending them all over the
> > > > network, simply how many more of them there are during the initial query.
> > > >
> > > > Does that make sense?
> > > >
> > > > I'm looking through the code trying to figure out where to add that
> > > > feature. I'm most of the way there I think but haven't found quite the
> > > > right place where all the terms are assembled yet.
> > > >
> > > > Thanks,
> > > > Kevin
> > > >
> > > > On Thursday, January 17, 2013 1:06:13 AM UTC-8, David Pilato wrote:
> > > >
> > > > > Hey Kevin,
> > > > >
> > > > > I probably did not understand your use case. Why the standard Term
> > > > > Facet did not answer to your need?
> > > > > There is an "other" field which is part of the answer:
> > > > >
> > > > > "facets" : {
> > > > > "f1" : {
> > > > > "\_type" : "terms",
> > > > > "missing" : 0,
> > > > > "total" : 3,
> > > > > "other" : 0,
> > > > > "terms" : [ {
> > > > > "term" : "b",
> > > > > "count" : 2
> > > > > }, {
> > > > > "term" : "c",
> > > > > "count" : 1
> > > > > } ]
> > > > > }
> > > > > }
> > > > > This field means that there are other terms than the TOP 10 (
> > > > > [https://github.com/\*\*elasticsearch/elasticsearch/\*\*issues/1029](https://github.com/ **elasticsearch/elasticsearch/** issues/1029)[https://github.com/elasticsearch/elasticsearch/issues/1029](https://github.com/elasticsearch/elasticsearch/issues/1029))
> > > > >
> > > > > Isn't it what you are after?
> > > > >
> > > > > David.
> > > > >
> > > > > Le 17 janvier 2013 à 00:25, Ivan Brusic [iv...@brusic.com](mailto:iv...@brusic.com) a écrit :
> > > > >
> > > > > The terms facets works off the termlist found in the Lucene index.
> > > > > There can never be a terms facet with a zero count. Facets never return the
> > > > > elements, just the term and the count.
> > > > >
> > > > > If you just want a list of terms, take a long at Jorg's plugin:
> > > > > [https://github.com/jprante/\*\*elasticsearch-index-termlist](https://github.com/jprante/**elasticsearch-index-termlist)[https://github.com/jprante/elasticsearch-index-termlist](https://github.com/jprante/elasticsearch-index-termlist)
> > > > >
> > > > > The Lucene code is found here:
> > > > > [https://github.com/jprante/\*\*elasticsearch-index-termlist/](https://github.com/jprante/**elasticsearch-index-termlist/)\*\*
> > > > > blob/master/src/main/java/org/ **elasticsearch/action/termlist/**
> > > > > TransportTermlistAction.java#\*\*L126[https://github.com/jprante/elasticsearch-index-termlist/blob/master/src/main/java/org/elasticsearch/action/termlist/TransportTermlistAction.java#L126](https://github.com/jprante/elasticsearch-index-termlist/blob/master/src/main/java/org/elasticsearch/action/termlist/TransportTermlistAction.java#L126)
> > > > >
> > > > > The plugin aggregates the termlists from each shard.
> > > > >
> > > > > --
> > > > > Ivan
> > > > >
> > > > > On Wed, Jan 16, 2013 at 2:46 PM, Kevin Fox [kfox...@gmail.com](mailto:kfox...@gmail.com)wrote:
> > > > >
> > > > > So, the all\_terms field says it returns all of the facets with counts
> > > > > = 0 even if there is nothing that matches that facet. That means, if there
> > > > > are a lot of them, it sends them all across the wire. What I'm trying to do
> > > > > is get the count of all facets minus the elements returned, so I can
> > > > > determine if I want to requery with something like all\_terms set to true or
> > > > > if it would be too expensive to do so. Any idea where in the code I could
> > > > > add that?
> > > > >
> > > > > Thanks,
> > > > > Kevin
> > > > >
> > > > > --
> > > > >
> > > > > --
> > > > > David Pilato
> > > > > [http://www.scrutmydocs.org/](http://www.scrutmydocs.org/)
> > > > > [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/)
> > > > > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
--
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 2:55am UTC](https://discuss.elastic.co/t/terms-facet-other-property-behavior-vs-documentation-question/10362/11 "2017-07-06T02:55:05Z")