# Facetted Search Misbehaving **URL:** \ **Category:** Elasticsearch\ **Created:** [April 20, 2014, 7:06pm UTC](https://discuss.elastic.co/t/facetted-search-misbehaving/17103 "2014-04-20T19:06:30Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Igal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igal/32/1119_2.png) [@Igal](https://discuss.elastic.co/u/Igal)\ **Post date:** [April 20, 2014, 7:06pm UTC](https://discuss.elastic.co/t/facetted-search-misbehaving/17103/1 "2014-04-20T19:06:30Z")
I have an index for a Store with a type for Items, i.e. /store/items. among other properties, Items have a Title (analyzed text), a Description (analyzed text), and Tags (not-analyzed text). I want to be able to show the facets over Tags with counts, so if a facet of the Tag "Yellow" has a count of 12, for example, then when the user adds that Tag to the filter she will see only 12 items. I am using a Filtered Query with Aggs, as shown below, on Elasticsearch 1.1.0 on a single node: GET \_search { "query": { "filtered": { "query": { "multi\_match": { "query": "Large Widgets" ,"fields": ["title^3", "description"] }} ,"filter": { "terms": { "tags": ["Colorful"] ,"execution": "and" }}}} ,"aggs": {"available\_tags": {"terms": {"field": "tags"}}, "size": 20} } I have two problems: 1. no matter what value I pass for the aggs/size I get 10 aggregations. I want to get more than 10. 2. the hits count that comes back when adding the new tag to the filter doesn't match the doc\_count that came with the aggregations, for example, the aggregations might show a doc\_count of 12 for the tag "Yellow", but if I add "Yellow" to the filter terms so that it reads "tags": ["Colorful", "Yellow"]I get 17 hits instead of the expected 12. am I doing something wrong? is there a bug somewhere? TIA, Igal -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/53541AB6.4000403%40getrailo.org](https://groups.google.com/d/msgid/elasticsearch/53541AB6.4000403%40getrailo.org). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\ **Post date:** [April 20, 2014, 7:32pm UTC](https://discuss.elastic.co/t/facetted-search-misbehaving/17103/2 "2014-04-20T19:32:31Z")
Sent from my iPhone > On Apr 20, 2014, at 3:06 PM, "Igal @ [getRailo.org](http://getRailo.org)" [igal@getrailo.org](mailto:igal@getrailo.org) wrote: > > I have an index for a Store with a type for Items, i.e. /store/items. among other properties, Items have a Title (analyzed text), a Description (analyzed text), and Tags (not-analyzed text). > > I want to be able to show the facets over Tags with counts, so if a facet of the Tag "Yellow" has a count of 12, for example, then when the user adds that Tag to the filter she will see only 12 items. > > I am using a Filtered Query with Aggs, as shown below, on Elasticsearch 1.1.0 on a single node: > > GET \_search { > "query": { > "filtered": { > "query": { > "multi\_match": { > "query": "Large Widgets" > ,"fields": ["title^3", "description"] > }} > ,"filter": { > "terms": { > "tags": ["Colorful"] > ,"execution": "and" > }}}} > ,"aggs": {"available\_tags": {"terms": {"field": "tags"}}, "size": 20} > } > > I have two problems: > > 1. no matter what value I pass for the aggs/size I get 10 aggregations. I want to get more than 10. > > 2. the hits count that comes back when adding the new tag to the filter doesn't match the doc\_count that came with the aggregations, for example, the aggregations might show a doc\_count of 12 for the tag "Yellow", but if I add "Yellow" to the filter terms so that it reads "tags": ["Colorful", "Yellow"]I get 17 hits instead of the expected 12. If I recall the terms filter is an or filter. So you are getting colorful or yellow. It might have an and option. Not sure, on my phone now. Otherwise use a bool filter with two must subfilters to do an and. Nik > am I doing something wrong? is there a bug somewhere? > > TIA, > > Igal > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/53541AB6.4000403%40getrailo.org](https://groups.google.com/d/msgid/elasticsearch/53541AB6.4000403%40getrailo.org). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/E9B277B9-31A2-4DC6-A4F9-A30E0C5265CF%40gmail.com](https://groups.google.com/d/msgid/elasticsearch/E9B277B9-31A2-4DC6-A4F9-A30E0C5265CF%40gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Igal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igal/32/1119_2.png) [@Igal](https://discuss.elastic.co/u/Igal)\ **Post date:** [April 20, 2014, 7:49pm UTC](https://discuss.elastic.co/t/facetted-search-misbehaving/17103/3 "2014-04-20T19:49:24Z")
hey Nik, thanks for your reply. you are correct about the default operator being "or", but as I've posted in the original post I explicitly set it to "and" via "execution": "and" and I see the difference in the results between "or" and "and" so I know that it is working. for some reason the numbers still do not match some of the time though. Igal On Sunday, April 20, 2014 12:32:31 PM UTC-7, Nikolas Everett wrote: > Sent from my iPhone > > > On Apr 20, 2014, at 3:06 PM, "Igal @ [getRailo.org](http://getRailo.org)" \<[ig...@getrailo.org](mailto:ig...@getrailo.org)\\> > > wrote: > > > > I have an index for a Store with a type for Items, i.e. /store/items. > > among other properties, Items have a Title (analyzed text), a Description > > (analyzed text), and Tags (not-analyzed text). > > > > I want to be able to show the facets over Tags with counts, so if a > > facet of the Tag "Yellow" has a count of 12, for example, then when the > > user adds that Tag to the filter she will see only 12 items. > > > > I am using a Filtered Query with Aggs, as shown below, on Elasticsearch > > 1.1.0 on a single node: > > > > GET \_search { > > "query": { > > "filtered": { > > "query": { > > "multi\_match": { > > "query": "Large Widgets" > > ,"fields": ["title^3", "description"] > > }} > > ,"filter": { > > "terms": { > > "tags": ["Colorful"] > > ,"execution": "and" > > }}}} > > ,"aggs": {"available\_tags": {"terms": {"field": "tags"}}, "size": 20} > > } > > > > I have two problems: > > > > 1. no matter what value I pass for the aggs/size I get 10 aggregations. > > I want to get more than 10. > > > > 2. the hits count that comes back when adding the new tag to the filter > > doesn't match the doc\_count that came with the aggregations, for example, > > the aggregations might show a doc\_count of 12 for the tag "Yellow", but if > > I add "Yellow" to the filter terms so that it reads "tags": [ "Colorful", > > "Yellow" ]I get 17 hits instead of the expected 12. > > If I recall the terms filter is an or filter. So you are getting colorful > or yellow. It might have an and option. Not sure, on my phone now. > Otherwise use a bool filter with two must subfilters to do an and. > > Nik > > > am I doing something wrong? is there a bug somewhere? > > > > TIA, > > > > Igal > > > > -- > > You received this message because you are subscribed to the Google > > Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/53541AB6.4000403%40getrailo.org](https://groups.google.com/d/msgid/elasticsearch/53541AB6.4000403%40getrailo.org). > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f6ce5c0d-37ec-45b6-972b-03dbd426e7de%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f6ce5c0d-37ec-45b6-972b-03dbd426e7de%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\ **Post date:** [April 20, 2014, 8:09pm UTC](https://discuss.elastic.co/t/facetted-search-misbehaving/17103/4 "2014-04-20T20:09:09Z")
Sorry, wasn't reading too closely. If the execution:and is taking sometimes then that sounds like a bug. Nik Sent from my iPhone > On Apr 20, 2014, at 3:49 PM, Igal [igal@getrailo.org](mailto:igal@getrailo.org) wrote: > > hey Nik, > > thanks for your reply. > > you are correct about the default operator being "or", but as I've posted in the original post I explicitly set it to "and" via "execution": "and" and I see the difference in the results between "or" and "and" so I know that it is working. for some reason the numbers still do not match some of the time though. > > Igal > > > On Sunday, April 20, 2014 12:32:31 PM UTC-7, Nikolas Everett wrote: > > > > Sent from my iPhone > > > > > On Apr 20, 2014, at 3:06 PM, "Igal @ [getRailo.org](http://getRailo.org)" [ig...@getrailo.org](mailto:ig...@getrailo.org) wrote: > > > > > > I have an index for a Store with a type for Items, i.e. /store/items. among other properties, Items have a Title (analyzed text), a Description (analyzed text), and Tags (not-analyzed text). > > > > > > I want to be able to show the facets over Tags with counts, so if a facet of the Tag "Yellow" has a count of 12, for example, then when the user adds that Tag to the filter she will see only 12 items. > > > > > > I am using a Filtered Query with Aggs, as shown below, on Elasticsearch 1.1.0 on a single node: > > > > > > GET \_search { > > > "query": { > > > "filtered": { > > > "query": { > > > "multi\_match": { > > > "query": "Large Widgets" > > > ,"fields": ["title^3", "description"] > > > }} > > > ,"filter": { > > > "terms": { > > > "tags": ["Colorful"] > > > ,"execution": "and" > > > }}}} > > > ,"aggs": {"available\_tags": {"terms": {"field": "tags"}}, "size": 20} > > > } > > > > > > I have two problems: > > > > > > 1. no matter what value I pass for the aggs/size I get 10 aggregations. I want to get more than 10. > > > > > > 2. the hits count that comes back when adding the new tag to the filter doesn't match the doc\_count that came with the aggregations, for example, the aggregations might show a doc\_count of 12 for the tag "Yellow", but if I add "Yellow" to the filter terms so that it reads "tags": ["Colorful", "Yellow"]I get 17 hits instead of the expected 12. > > > > If I recall the terms filter is an or filter. So you are getting colorful or yellow. It might have an and option. Not sure, on my phone now. Otherwise use a bool filter with two must subfilters to do an and. > > > > Nik > > > > > am I doing something wrong? is there a bug somewhere? > > > > > > TIA, > > > > > > Igal > > > > > > -- > > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/53541AB6.4000403%40getrailo.org](https://groups.google.com/d/msgid/elasticsearch/53541AB6.4000403%40getrailo.org). > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f6ce5c0d-37ec-45b6-972b-03dbd426e7de%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f6ce5c0d-37ec-45b6-972b-03dbd426e7de%40googlegroups.com). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/B4C0631C-7665-4380-BDD6-E6C84292CAEB%40gmail.com](https://groups.google.com/d/msgid/elasticsearch/B4C0631C-7665-4380-BDD6-E6C84292CAEB%40gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Igal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igal/32/1119_2.png) [@Igal](https://discuss.elastic.co/u/Igal)\ **Post date:** [April 21, 2014, 6:36pm UTC](https://discuss.elastic.co/t/facetted-search-misbehaving/17103/5 "2014-04-21T18:36:25Z")
Thanks Nik -- that's what I was thinking too. I was wondering if someone from the ES team could confirm that my code is correct for what I'm trying to achieve. Igal On 4/20/2014 1:09 PM, Nik Everett wrote: > Sorry, wasn't reading too closely. If the execution:and is taking > sometimes then that sounds like a bug. > > Nik > > Sent from my iPhone > > On Apr 20, 2014, at 3:49 PM, Igal \<[igal@getrailo.org](mailto:igal@getrailo.org) > [mailto:igal@getrailo.org](mailto:igal@getrailo.org)\> wrote: > > > hey Nik, > > > > thanks for your reply. > > > > you are correct about the default operator being "or", but as I've > > posted in the original post I explicitly set it to "and" via > > "execution": "and" and I see the difference in the results between > > "or" and "and" so I know that it is working. for some reason the > > numbers still do not match some of the time though. > > > > Igal > > > > On Sunday, April 20, 2014 12:32:31 PM UTC-7, Nikolas Everett wrote: > > > > ``` > > Sent from my iPhone > > > > > On Apr 20, 2014, at 3:06 PM, "Igal @ getRailo.org > > " > wrote: > > > > > > I have an index for a Store with a type for Items, i.e. > > /store/items. among other properties, Items have a Title > > (analyzed text), a Description (analyzed text), and Tags > > (not-analyzed text). > > > > > > I want to be able to show the facets over Tags with counts, so > > if a facet of the Tag "Yellow" has a count of 12, for example, > > then when the user adds that Tag to the filter she will see only > > 12 items. > > > > > > I am using a Filtered Query with Aggs, as shown below, on > > Elasticsearch 1.1.0 on a single node: > > > > > > GET _search { > > > "query": { > > > "filtered": { > > > "query": { > > > "multi_match": { > > > "query": "Large Widgets" > > > ,"fields": ["title^3", "description"] > > > }} > > > ,"filter": { > > > "terms": { > > > "tags": ["Colorful"] > > > ,"execution": "and" > > > }}}} > > > ,"aggs": {"available_tags": {"terms": {"field": "tags"}}, > > "size": 20} > > > } > > > > > > I have two problems: > > > > > > 1) no matter what value I pass for the aggs/size I get 10 > > aggregations. I want to get more than 10. > > > > > > 2) the hits count that comes back when adding the new tag to > > the filter doesn't match the doc_count that came with the > > aggregations, for example, the aggregations might show a > > doc_count of 12 for the tag "Yellow", but if I add "Yellow" to > > the filter terms so that it reads "tags": [ "Colorful", "Yellow" > > ]I get 17 hits instead of the expected 12. > > > > If I recall the terms filter is an or filter. So you are getting > > colorful or yellow. It might have an and option. Not sure, on my > > phone now. Otherwise use a bool filter with two must subfilters > > to do an and. > > > > Nik > > > > > > am I doing something wrong? is there a bug somewhere? > > > > > > TIA, > > > > > > > > > Igal > > > > > > > > > > > > > > > -- > > > You received this message because you are subscribed to the > > Google Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from > > it, send an email to elasticsearc...@googlegroups.com . > > > To view this discussion on the web visit > > https://groups.google.com/d/msgid/elasticsearch/53541AB6.4000403%40getrailo.org > > . > > > > > For more options, visit https://groups.google.com/d/optout > > . > > > > ``` > > > > -- > > You received this message because you are subscribed to the Google > > Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, > > send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) > > [mailto:elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/f6ce5c0d-37ec-45b6-972b-03dbd426e7de%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f6ce5c0d-37ec-45b6-972b-03dbd426e7de%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/f6ce5c0d-37ec-45b6-972b-03dbd426e7de%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f6ce5c0d-37ec-45b6-972b-03dbd426e7de%40googlegroups.com?utm_medium=email&utm_source=footer). > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/xYRaJa04wfc/unsubscribe](https://groups.google.com/d/topic/elasticsearch/xYRaJa04wfc/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) > > [mailto:elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/B4C0631C-7665-4380-BDD6-E6C84292CAEB%40gmail.com](https://groups.google.com/d/msgid/elasticsearch/B4C0631C-7665-4380-BDD6-E6C84292CAEB%40gmail.com) > > [https://groups.google.com/d/msgid/elasticsearch/B4C0631C-7665-4380-BDD6-E6C84292CAEB%40gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/B4C0631C-7665-4380-BDD6-E6C84292CAEB%40gmail.com?utm_medium=email&utm_source=footer). > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- Igal Sapir Railo Core Developer > **[Top 10 Best Bitcoin Slots Sites | Best Online Casino 2023](https://www.getrailo.org/)** > > Compare the Best Bitcoin Slots Sites now at getrailo.org. Slots represent some of the most popular and exciting online casino offerings. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/53556529.7010709%40getrailo.org](https://groups.google.com/d/msgid/elasticsearch/53556529.7010709%40getrailo.org). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Igal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igal/32/1119_2.png) [@Igal](https://discuss.elastic.co/u/Igal)\ **Post date:** [April 24, 2014, 5:55pm UTC](https://discuss.elastic.co/t/facetted-search-misbehaving/17103/6 "2014-04-24T17:55:50Z")
I noticed that the problem usually happens when the number of results in the agg is rather small. I have 5 shards on that index, and the aggs return 10 results (again, I'm not sure why not more when I request specifically as mentioned in item 1 in the OP. it looks like when the agg value is less than 50 (5 shards \* 10 aggs) I get this problem, so the aggs shows a smaller value than the actual hits returned. is that a known issue? any ideas on how I can get it to work properly? I also posted to [lucene - Faceted Search and Aggregations Do Not Work Properly - Stack Overflow](http://stackoverflow.com/questions/23254153/faceted-search-and-aggregations-do-not-work-properly) since there was no much response here. thanks, Igal On Monday, April 21, 2014 11:36:25 AM UTC-7, Igal wrote: > Thanks Nik -- that's what I was thinking too. > > I was wondering if someone from the ES team could confirm that my code is > correct for what I'm trying to achieve. > > Igal > > On 4/20/2014 1:09 PM, Nik Everett wrote: > > Sorry, wasn't reading too closely. If the execution:and is taking > sometimes then that sounds like a bug. > > Nik > > Sent from my iPhone > > On Apr 20, 2014, at 3:49 PM, Igal [igal@getrailo.org](mailto:igal@getrailo.org) wrote: > > hey Nik, > > thanks for your reply. > > you are correct about the default operator being "or", but as I've > posted in the original post I explicitly set it to "and" via "execution": > "and" and I see the difference in the results between "or" and "and" so I > know that it is working. for some reason the numbers still do not match > some of the time though. > > Igal > > On Sunday, April 20, 2014 12:32:31 PM UTC-7, Nikolas Everett wrote: > > > Sent from my iPhone > > > > > On Apr 20, 2014, at 3:06 PM, "Igal @ [getRailo.org](http://getRailo.org)" [ig...@getrailo.org](mailto:ig...@getrailo.org) > > > wrote: > > > > > > I have an index for a Store with a type for Items, i.e. /store/items. > > > among other properties, Items have a Title (analyzed text), a Description > > > (analyzed text), and Tags (not-analyzed text). > > > > > > I want to be able to show the facets over Tags with counts, so if a > > > facet of the Tag "Yellow" has a count of 12, for example, then when the > > > user adds that Tag to the filter she will see only 12 items. > > > > > > I am using a Filtered Query with Aggs, as shown below, on Elasticsearch > > > 1.1.0 on a single node: > > > > > > GET \_search { > > > "query": { > > > "filtered": { > > > "query": { > > > "multi\_match": { > > > "query": "Large Widgets" > > > ,"fields": ["title^3", "description"] > > > }} > > > ,"filter": { > > > "terms": { > > > "tags": ["Colorful"] > > > ,"execution": "and" > > > }}}} > > > ,"aggs": {"available\_tags": {"terms": {"field": "tags"}}, "size": 20} > > > } > > > > > > I have two problems: > > > > > > 1. no matter what value I pass for the aggs/size I get 10 aggregations. > > > I want to get more than 10. > > > > > > 2. the hits count that comes back when adding the new tag to the filter > > > doesn't match the doc\_count that came with the aggregations, for example, > > > the aggregations might show a doc\_count of 12 for the tag "Yellow", but if > > > I add "Yellow" to the filter terms so that it reads "tags": [ "Colorful", > > > "Yellow" ]I get 17 hits instead of the expected 12. > > > > If I recall the terms filter is an or filter. So you are getting colorful > > or yellow. It might have an and option. Not sure, on my phone now. > > Otherwise use a bool filter with two must subfilters to do an and. > > > > Nik > > > > > am I doing something wrong? is there a bug somewhere? > > > > > > TIA, > > > > > > Igal > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/53541AB6.4000403%40getrailo.org](https://groups.google.com/d/msgid/elasticsearch/53541AB6.4000403%40getrailo.org). > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/f6ce5c0d-37ec-45b6-972b-03dbd426e7de%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f6ce5c0d-37ec-45b6-972b-03dbd426e7de%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/f6ce5c0d-37ec-45b6-972b-03dbd426e7de%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f6ce5c0d-37ec-45b6-972b-03dbd426e7de%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/xYRaJa04wfc/unsubscribe](https://groups.google.com/d/topic/elasticsearch/xYRaJa04wfc/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/B4C0631C-7665-4380-BDD6-E6C84292CAEB%40gmail.com](https://groups.google.com/d/msgid/elasticsearch/B4C0631C-7665-4380-BDD6-E6C84292CAEB%40gmail.com)[https://groups.google.com/d/msgid/elasticsearch/B4C0631C-7665-4380-BDD6-E6C84292CAEB%40gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/B4C0631C-7665-4380-BDD6-E6C84292CAEB%40gmail.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > Igal Sapir > Railo Core Developerhttp://getRailo.org/ -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/b43fd972-ae9c-4368-9f4e-2428d71eb91f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b43fd972-ae9c-4368-9f4e-2428d71eb91f%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Igal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igal/32/1119_2.png) [@Igal](https://discuss.elastic.co/u/Igal)\ **Post date:** [May 3, 2014, 5:33pm UTC](https://discuss.elastic.co/t/facetted-search-misbehaving/17103/7 "2014-05-03T17:33:54Z")
for whoever finds this thread in the future, see > > > I'm working with nested documents and have noticed that my faceted search interf…ace is giving the wrong counts when I have more than one shard. To be more specific, I'm working with RDF triples (entity \> attribute \> value) and I'm nesting the attributes (called predicates in my example): > > \`\`\` > { > "\_id" : "512a2c022f0b4e3daa341e6c8bcf6c2f", > "url": "http://dbpedia.org/resource/Alan\_Shepard", > "predicates": \[ > { > "type": "type", > "string\_value": \["thing", "person", "astronaut"\] > }, { > "type": "label", > "string\_value": \["Alan Shepard"\] > }, { > "type": "time in space", > "float\_value": \[216.950\] > }, > ... lots more > \] > } > \`\`\` > > I've created a shell script (https://gist.github.com/1196986) that recreates the problem with a fresh index. The created data set has these totals: > \- thing (30) > \- creative work (20) > \- video game (10) > \- tv show (10) > \- people (10) > > With only \*\*one shard\*\* the following query gives the correct counts no matter what the size parameter is set to: > > \`\`\` > { > "size": 0, > "query": { > "match\_all": {} > }, > "facets": { > "type\_counts": { > "terms": { > "field": "string\_value", > "size": 5 > }, > "nested": "predicates", > "facet\_filter": { > "term": { > "type": "type" > } > } > } > } > } > \`\`\` > > However, with \*\*more than one shard\*\* the size parameter affects the accuracy of the counts. If it is equal to or greater than the number of terms returned by the facet query (5 in this case) then it works fine. However, the terms at the bottom of the list start to display low counts as you reduce the size parameter: > > With "size" : 4 > \- thing (30) > \- creative work (20) > \- video game (10) > \- \*\*tv show (9)\*\* > > With "size" : 3 > \- thing (30) > \- \*\*creative work (15)\*\* > \- \*\*video game (9)\*\* > > With "size" : 2 > \- thing (30) > \- \*\*creative work (15)\*\* > > So it looks like the sub-totals from some of the shards aren't being included for some reason. BTW I'm on ubuntu and the problem seems to affect all versions of ES I've tried (17.0, 17.1 and 17.6). Any ideas...? > > P.S. absolutely loving ES - it's made my life a lot easier :) On 4/24/2014 10:55 AM, Igal wrote: > I noticed that the problem usually happens when the number of results > in the agg is rather small. > > I have 5 shards on that index, and the aggs return 10 results (again, > I'm not sure why not more when I request specifically as mentioned in > item 1 in the OP. > > it looks like when the agg value is less than 50 (5 shards \* 10 aggs) > I get this problem, so the aggs shows a smaller value than the actual > hits returned. > > is that a known issue? any ideas on how I can get it to work properly? > > I also posted > to [lucene - Faceted Search and Aggregations Do Not Work Properly - Stack Overflow](http://stackoverflow.com/questions/23254153/faceted-search-and-aggregations-do-not-work-properly) > since there was no much response here. > > thanks, > > Igal > > On Monday, April 21, 2014 11:36:25 AM UTC-7, Igal wrote: > > ``` > Thanks Nik -- that's what I was thinking too. > > I was wondering if someone from the ES team could confirm that my > code is correct for what I'm trying to achieve. > > Igal > > On 4/20/2014 1:09 PM, Nik Everett wrote: > > ``` > > > ``` > > Sorry, wasn't reading too closely. If the execution:and is taking > > sometimes then that sounds like a bug. > > > > Nik > > > > Sent from my iPhone > > > > On Apr 20, 2014, at 3:49 PM, Igal > > wrote: > > > > ``` > > > > > ``` > > > hey Nik, > > > > > > thanks for your reply. > > > > > > you are correct about the default operator being "or", but as > > > I've posted in the original post I explicitly set it to "and" > > > via "execution": "and" and I see the difference in the results > > > between "or" and "and" so I know that it is working. for some > > > reason the numbers still do not match some of the time though. > > > > > > Igal > > > > > > On Sunday, April 20, 2014 12:32:31 PM UTC-7, Nikolas Everett wrote: > > > > > > Sent from my iPhone > > > > > > > On Apr 20, 2014, at 3:06 PM, "Igal @ getRailo.org > > > " wrote: > > > > > > > > I have an index for a Store with a type for Items, i.e. > > > /store/items. among other properties, Items have a Title > > > (analyzed text), a Description (analyzed text), and Tags > > > (not-analyzed text). > > > > > > > > I want to be able to show the facets over Tags with > > > counts, so if a facet of the Tag "Yellow" has a count of 12, > > > for example, then when the user adds that Tag to the filter > > > she will see only 12 items. > > > > > > > > I am using a Filtered Query with Aggs, as shown below, on > > > Elasticsearch 1.1.0 on a single node: > > > > > > > > GET _search { > > > > "query": { > > > > "filtered": { > > > > "query": { > > > > "multi_match": { > > > > "query": "Large Widgets" > > > > ,"fields": ["title^3", "description"] > > > > }} > > > > ,"filter": { > > > > "terms": { > > > > "tags": ["Colorful"] > > > > ,"execution": "and" > > > > }}}} > > > > ,"aggs": {"available_tags": {"terms": {"field": "tags"}}, > > > "size": 20} > > > > } > > > > > > > > I have two problems: > > > > > > > > 1) no matter what value I pass for the aggs/size I get 10 > > > aggregations. I want to get more than 10. > > > > > > > > 2) the hits count that comes back when adding the new tag > > > to the filter doesn't match the doc_count that came with the > > > aggregations, for example, the aggregations might show a > > > doc_count of 12 for the tag "Yellow", but if I add "Yellow" > > > to the filter terms so that it reads "tags": [ "Colorful", > > > "Yellow" ]I get 17 hits instead of the expected 12. > > > > > > If I recall the terms filter is an or filter. So you are > > > getting colorful or yellow. It might have an and option. Not > > > sure, on my phone now. Otherwise use a bool filter with two > > > must subfilters to do an and. > > > > > > Nik > > > > > > > > am I doing something wrong? is there a bug somewhere? > > > > > > > > TIA, > > > > > > > > > > > > Igal > > > > > > > > > > > > > > > > > > > > -- > > > > You received this message because you are subscribed to > > > the Google Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails > > > from it, send an email to elasticsearc...@googlegroups.com. > > > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/elasticsearch/53541AB6.4000403%40getrailo.org > > > . > > > > > > > For more options, visit https://groups.google.com/d/optout > > > . > > > > > > -- > > > You received this message because you are subscribed to the > > > Google Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from > > > it, send an email to elasticsearch+unsubscribe@googlegroups.com > > > . > > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/elasticsearch/f6ce5c0d-37ec-45b6-972b-03dbd426e7de%40googlegroups.com > > > . > > > For more options, visit https://groups.google.com/d/optout > > > . > > > > > > ``` > > > > ``` > > -- > > You received this message because you are subscribed to a topic > > in the Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > https://groups.google.com/d/topic/elasticsearch/xYRaJa04wfc/unsubscribe > > . > > To unsubscribe from this group and all its topics, send an email > > to elasticsearch+unsubscribe@googlegroups.com > > . > > To view this discussion on the web visit > > https://groups.google.com/d/msgid/elasticsearch/B4C0631C-7665-4380-BDD6-E6C84292CAEB%40gmail.com > > . > > For more options, visit https://groups.google.com/d/optout > > . > > > > ``` > > ``` > -- > Igal Sapir > Railo Core Developer > http://getRailo.org/ > > ``` > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/xYRaJa04wfc/unsubscribe](https://groups.google.com/d/topic/elasticsearch/xYRaJa04wfc/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) > [mailto:elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/b43fd972-ae9c-4368-9f4e-2428d71eb91f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b43fd972-ae9c-4368-9f4e-2428d71eb91f%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/b43fd972-ae9c-4368-9f4e-2428d71eb91f%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/b43fd972-ae9c-4368-9f4e-2428d71eb91f%40googlegroups.com?utm_medium=email&utm_source=footer). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- Igal Sapir Railo Core Developer > **[Top 10 Best Bitcoin Slots Sites | Best Online Casino 2023](https://www.getrailo.org/)** > > Compare the Best Bitcoin Slots Sites now at getrailo.org. Slots represent some of the most popular and exciting online casino offerings. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/53652882.4040406%40getrailo.org](https://groups.google.com/d/msgid/elasticsearch/53652882.4040406%40getrailo.org). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Igal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igal/32/1119_2.png) [@Igal](https://discuss.elastic.co/u/Igal)\ **Post date:** [May 3, 2014, 10:40pm UTC](https://discuss.elastic.co/t/facetted-search-misbehaving/17103/8 "2014-05-03T22:40:50Z")
looks like the "size" issue was my mistake. I've placed it in the "available\_tags" scope instead of the "terms" scope. so the line for the aggs in the request should read instead: ``` ,"aggs": {"available_tags": {"terms": {"field": "tags", "size": 20}}} ``` On 5/3/2014 10:33 AM, Igal @ [getRailo.org](http://getRailo.org) wrote: > for whoever finds this thread in the future, see > [terms facet gives wrong count with n\_shards \> 1 · Issue #1305 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/1305) > > On 4/24/2014 10:55 AM, Igal wrote: > > > I noticed that the problem usually happens when the number of results > > in the agg is rather small. > > > > I have 5 shards on that index, and the aggs return 10 results (again, > > I'm not sure why not more when I request specifically as mentioned in > > item 1 in the OP. > > > > it looks like when the agg value is less than 50 (5 shards \* 10 aggs) > > I get this problem, so the aggs shows a smaller value than the actual > > hits returned. > > > > is that a known issue? any ideas on how I can get it to work properly? > > > > I also posted to > > [lucene - Faceted Search and Aggregations Do Not Work Properly - Stack Overflow](http://stackoverflow.com/questions/23254153/faceted-search-and-aggregations-do-not-work-properly) > > since there was no much response here. > > > > thanks, > > > > Igal > > > > On Monday, April 21, 2014 11:36:25 AM UTC-7, Igal wrote: > > > > ``` > > Thanks Nik -- that's what I was thinking too. > > > > I was wondering if someone from the ES team could confirm that my > > code is correct for what I'm trying to achieve. > > > > Igal > > > > On 4/20/2014 1:09 PM, Nik Everett wrote: > > > > ``` > > > > > ``` > > > Sorry, wasn't reading too closely. If the execution:and is > > > taking sometimes then that sounds like a bug. > > > > > > Nik > > > > > > Sent from my iPhone > > > > > > On Apr 20, 2014, at 3:49 PM, Igal > > > wrote: > > > > > > ``` > > > > > > > ``` > > > > hey Nik, > > > > > > > > thanks for your reply. > > > > > > > > you are correct about the default operator being "or", but as > > > > I've posted in the original post I explicitly set it to "and" > > > > via "execution": "and" and I see the difference in the results > > > > between "or" and "and" so I know that it is working. for some > > > > reason the numbers still do not match some of the time though. > > > > > > > > Igal > > > > > > > > On Sunday, April 20, 2014 12:32:31 PM UTC-7, Nikolas Everett > > > > wrote: > > > > > > > > Sent from my iPhone > > > > > > > > > On Apr 20, 2014, at 3:06 PM, "Igal @ getRailo.org > > > > " wrote: > > > > > > > > > > I have an index for a Store with a type for Items, i.e. > > > > /store/items. among other properties, Items have a Title > > > > (analyzed text), a Description (analyzed text), and Tags > > > > (not-analyzed text). > > > > > > > > > > I want to be able to show the facets over Tags with > > > > counts, so if a facet of the Tag "Yellow" has a count of > > > > 12, for example, then when the user adds that Tag to the > > > > filter she will see only 12 items. > > > > > > > > > > I am using a Filtered Query with Aggs, as shown below, on > > > > Elasticsearch 1.1.0 on a single node: > > > > > > > > > > GET _search { > > > > > "query": { > > > > > "filtered": { > > > > > "query": { > > > > > "multi_match": { > > > > > "query": "Large Widgets" > > > > > ,"fields": ["title^3", "description"] > > > > > }} > > > > > ,"filter": { > > > > > "terms": { > > > > > "tags": ["Colorful"] > > > > > ,"execution": "and" > > > > > }}}} > > > > > ,"aggs": {"available_tags": {"terms": {"field": > > > > "tags"}}, "size": 20} > > > > > } > > > > > > > > > > I have two problems: > > > > > > > > > > 1) no matter what value I pass for the aggs/size I get 10 > > > > aggregations. I want to get more than 10. > > > > > > > > > > 2) the hits count that comes back when adding the new tag > > > > to the filter doesn't match the doc_count that came with > > > > the aggregations, for example, the aggregations might show > > > > a doc_count of 12 for the tag "Yellow", but if I add > > > > "Yellow" to the filter terms so that it reads "tags": [ > > > > "Colorful", "Yellow" ]I get 17 hits instead of the expected > > > > 12. > > > > > > > > If I recall the terms filter is an or filter. So you are > > > > getting colorful or yellow. It might have an and option. > > > > Not sure, on my phone now. Otherwise use a bool filter with > > > > two must subfilters to do an and. > > > > > > > > Nik > > > > > > > > > > am I doing something wrong? is there a bug somewhere? > > > > > > > > > > TIA, > > > > > > > > > > > > > > > Igal > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > You received this message because you are subscribed to > > > > the Google Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails > > > > from it, send an email to elasticsearc...@googlegroups.com. > > > > > To view this discussion on the web visit > > > > https://groups.google.com/d/msgid/elasticsearch/53541AB6.4000403%40getrailo.org > > > > . > > > > > > > > > For more options, visit > > > > https://groups.google.com/d/optout > > > > . > > > > > > > > -- > > > > You received this message because you are subscribed to the > > > > Google Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from > > > > it, send an email to elasticsearch+unsubscribe@googlegroups.com > > > > . > > > > To view this discussion on the web visit > > > > https://groups.google.com/d/msgid/elasticsearch/f6ce5c0d-37ec-45b6-972b-03dbd426e7de%40googlegroups.com > > > > . > > > > For more options, visit https://groups.google.com/d/optout > > > > . > > > > > > > > ``` > > > > > > ``` > > > -- > > > You received this message because you are subscribed to a topic > > > in the Google Groups "elasticsearch" group. > > > To unsubscribe from this topic, visit > > > https://groups.google.com/d/topic/elasticsearch/xYRaJa04wfc/unsubscribe > > > . > > > To unsubscribe from this group and all its topics, send an email > > > to elasticsearch+unsubscribe@googlegroups.com > > > . > > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/elasticsearch/B4C0631C-7665-4380-BDD6-E6C84292CAEB%40gmail.com > > > . > > > For more options, visit https://groups.google.com/d/optout > > > . > > > > > > ``` > > > > ``` > > -- > > Igal Sapir > > Railo Core Developer > > http://getRailo.org/ > > > > ``` > > > > -- > > You received this message because you are subscribed to a topic in > > the Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/xYRaJa04wfc/unsubscribe](https://groups.google.com/d/topic/elasticsearch/xYRaJa04wfc/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) > > [mailto:elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/b43fd972-ae9c-4368-9f4e-2428d71eb91f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b43fd972-ae9c-4368-9f4e-2428d71eb91f%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/b43fd972-ae9c-4368-9f4e-2428d71eb91f%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/b43fd972-ae9c-4368-9f4e-2428d71eb91f%40googlegroups.com?utm_medium=email&utm_source=footer). > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > Igal Sapir > Railo Core Developer > [http://getRailo.org/](http://getRailo.org/) -- Igal Sapir Railo Core Developer > **[Top 10 Best Bitcoin Slots Sites | Best Online Casino 2023](https://www.getrailo.org/)** > > Compare the Best Bitcoin Slots Sites now at getrailo.org. Slots represent some of the most popular and exciting online casino offerings. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/53657072.8090807%40getrailo.org](https://groups.google.com/d/msgid/elasticsearch/53657072.8090807%40getrailo.org). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:31am UTC](https://discuss.elastic.co/t/facetted-search-misbehaving/17103/9 "2017-07-06T01:31:58Z")