# Filter, facets and filtered query **URL:** \ **Category:** Elasticsearch\ **Created:** [March 29, 2011, 9:14pm UTC](https://discuss.elastic.co/t/filter-facets-and-filtered-query/4169 "2011-03-29T21:14:11Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![ruflin\_2](https://avatars.discourse-cdn.com/v4/letter/r/a8b319/32.png) [@ruflin\_2](https://discuss.elastic.co/u/ruflin_2)\ **Post date:** [March 29, 2011, 9:14pm UTC](https://discuss.elastic.co/t/filter-facets-and-filtered-query/4169/1 "2011-03-29T21:14:11Z")
Hi I'm refactoring the elastica client at the moment. So I realized that I'm not completely sure how filters, facets and filtered queries can be combined. Lets take the last example in the link below: [http://www.elasticsearch.org/guide/reference/api/search/filter.html](http://www.elasticsearch.org/guide/reference/api/search/filter.html) We have a query, a filter and a facet. Now I have several questions: - Would it be possible to use a filtered query instead of a "standard" query? - Could I use in addition a facet\_filter? - When should I use a filtered query or a query in combination with a filter? ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [March 30, 2011, 8:50am UTC](https://discuss.elastic.co/t/filter-facets-and-filtered-query/4169/2 "2011-03-30T08:50:21Z")
Heya, inlined: On Tuesday, March 29, 2011 at 11:14 PM, ruflin wrote: > Hi > > I'm refactoring the elastica client at the moment. So I realized that > I'm not completely sure how filters, facets and filtered queries can > be combined. Lets take the last example in the link below: > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/search/filter.html) > > We have a query, a filter and a facet. Now I have several questions: > > - Would it be possible to use a filtered query instead of a "standard" > query? > Yea, filtered query is just another query type, like term query. > - Could I use in addition a facet\_filter? > Sure, you can place a facet\_filter in each facet, accepting any type of the different filters provided. > - When should I use a filtered query or a query in combination with a > filter? > Depends what you are after. If you don't use facets, then a filtered query might be better perf wise compared to a query and a search filter (though difference will usually be very small). The filter element is there to allow for simplified facet based navigation. ---
**Author:** ![ruflin\_2](https://avatars.discourse-cdn.com/v4/letter/r/a8b319/32.png) [@ruflin\_2](https://discuss.elastic.co/u/ruflin_2)\ **Post date:** [March 30, 2011, 9:10am UTC](https://discuss.elastic.co/t/filter-facets-and-filtered-query/4169/3 "2011-03-30T09:10:17Z")
Thanks for your answer. Now it makes much more sense. I did not realize that filtered is just another query type. On Mar 30, 10:50 am, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > Heya, inlined:On Tuesday, March 29, 2011 at 11:14 PM, ruflin wrote: > > > Hi > > > I'm refactoring the elastica client at the moment. So I realized that > > I'm not completely sure how filters, facets and filtered queries can > > be combined. Lets take the last example in the link below: > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/search/filter.html) > > > We have a query, a filter and a facet. Now I have several questions: > > > > - Would it be possible to use a filtered query instead of a "standard" > > query? > > Yea, filtered query is just another query type, like term query.\> - Could I use in addition a facet\_filter? > > Sure, you can place a facet\_filter in each facet, accepting any type of the different filters provided. > > > - When should I use a filtered query or a query in combination with a > > filter? > > Depends what you are after. If you don't use facets, then a filtered query might be better perf wise compared to a query and a search filter (though difference will usually be very small). The filter element is there to allow for simplified facet based navigation. ---
**Author:** ![melix](https://avatars.discourse-cdn.com/v4/letter/m/b38774/32.png) [@melix](https://discuss.elastic.co/u/melix)\ **Post date:** [July 7, 2011, 7:41am UTC](https://discuss.elastic.co/t/filter-facets-and-filtered-query/4169/4 "2011-07-07T07:41:36Z")
I'm also confused about this filter/facet combination. I can see why you would want to add specific filters to facets, but I don't understand why the global filter is not used to compute facets. For example, if you do : { "query": ..., "filter": ..., "facets": ... } Then the facet counts are returned without taking into account the filter. This seems illogic to me, as the documentation states that the facets are by default bound to the current query. Either the documentation should be more precise, telling that the filter is ignored, or there should be an option to make the filter active in facets too (which I think would be better for both backward compatibility and usability). ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [July 7, 2011, 9:17am UTC](https://discuss.elastic.co/t/filter-facets-and-filtered-query/4169/5 "2011-07-07T09:17:29Z")
Hi Cedric On Thu, 2011-07-07 at 00:41 -0700, melix wrote: > I'm also confused about this filter/facet combination. I can see why you > would want to add specific filters to facets, but I don't understand why the > global filter is not used to compute facets. For example, if you do : > { "query": ..., > "filter": ..., > "facets": ... } > > Then the facet counts are returned without taking into account the filter. > This seems illogic to me, as the documentation states that the facets are by > default bound to the current query. Either the documentation should be more > precise, telling that the filter is ignored, or there should be an option to > make the filter active in facets too (which I think would be better for both > backward compatibility and usability). "the facets are by default bound to the current query" This means the query that is specified in the "query" parameter. Normally, if you want a filtered query, you do: { "query": { "filtered": { "query": { find foo }, "filter": { filter by bar } }, "facets": { ... } # foo filtered by bar } Facets will be bound by the filter specified in that query. The TOP LEVEL filter parameter was added so that you can get your facet results on all documents matching the query { find foo }, then later filter your query results with { filter by bar }, which would look like: { "query": { find foo }, "filter": { filter by bar }, "facets": { ... } # foo, not filtered } clint ---
**Author:** ![melix](https://avatars.discourse-cdn.com/v4/letter/m/b38774/32.png) [@melix](https://discuss.elastic.co/u/melix)\ **Post date:** [July 7, 2011, 9:29am UTC](https://discuss.elastic.co/t/filter-facets-and-filtered-query/4169/6 "2011-07-07T09:29:37Z")
I understand, but in a facetted search, I would expect everything to be refined, so the facet counts should IMHO change whenever I add a filter. I still think an option to make have the global filter used by the facets would be nice 🙂 > Facets will be bound by the filter specified in that query. > > The TOP LEVEL filter parameter was added so that you can get your facet > results on all documents matching the query { find foo }, then later > filter your query results with { filter by bar }, which would look like: > > { > "query": { find foo }, > "filter": { filter by bar }, > "facets": { ... } # foo, not filtered > } ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [July 7, 2011, 9:38am UTC](https://discuss.elastic.co/t/filter-facets-and-filtered-query/4169/7 "2011-07-07T09:38:47Z")
On Thu, 2011-07-07 at 11:29 +0200, Cédric Champeau wrote: > I understand, but in a facetted search, I would expect everything to be > refined, so the facet counts should IMHO change whenever I add a filter. > I still think an option to make have the global filter used by the > facets would be nice 🙂 Cedric, I think you missed my point. The top-level filter is there ONLY for situations where you specifically DON'T want the facets to be bound by your filter. So if you use the filtered query, then it will do exactly what you want. clint ---
**Author:** ![melix](https://avatars.discourse-cdn.com/v4/letter/m/b38774/32.png) [@melix](https://discuss.elastic.co/u/melix)\ **Post date:** [July 7, 2011, 9:58am UTC](https://discuss.elastic.co/t/filter-facets-and-filtered-query/4169/8 "2011-07-07T09:58:10Z")
Sure, but I think you also miss some background about how my queries are generated. In fact, I have a legacy application which has its own query format. I am able to map this query format to various search engines, including Lucene, MG4J and now I am working on Elasticsearch. This abstract query format is quite old and while I am able to map it to various search engines, it's logic is sometimes slightly different from the underlying search engine. This abstract model does have a notion of filter on a query, and I could use it, but there is another problem : The second point is about the query format. My abstract query model is an object model. The input format is parsed into this abstract model. I had XML until now, and I'm adding a JSON format now. I posted a message yesterday regarding the ability to use the internal JSON parsers from ES to be able to reuse the Elasticsearch syntax at some points. Unfortunately, this is not possible unless using the "extraSource" parameter of a query. Having those two points in mind, I have a choice to make : ``` - add the ability to use the ElasticSearch JSON syntax of a query ``` filter inside my own JSON query format, but it would require me to write parsers for the various filters ES supports - add an "extraSource" section into my custom JSON format where the user may add a filter. This is the solution I managed to implement. While it has the restriction of not being able to add a filter on every subquery, it is perfectly acceptable. However, the problem with this solution is that the user must use a boolean AND query in order to emulate a filter at the query level to have the filters "used" by the facets. Indeed, this is not a filter and suffers performance penalties. Having a flag allowing the "global filter" to be used by facets would solve this issue for me. The alternative, reimplementing json parsers where I wanted the very same format as Elasticsearch to be supported, looks unproductive. I hope this makes it clearer why I think this flag would be useful 😃 Le 07/07/2011 11:38, Clinton Gormley a écrit : > On Thu, 2011-07-07 at 11:29 +0200, Cédric Champeau wrote: > > > I understand, but in a facetted search, I would expect everything to be > > refined, so the facet counts should IMHO change whenever I add a filter. > > I still think an option to make have the global filter used by the > > facets would be nice 🙂 > > Cedric, I think you missed my point. The top-level filter is there ONLY > > for situations where you specifically DON'T want the facets to be bound > > by your filter. > > So if you use the filtered query, then it will do exactly what you want. > > clint ---
**Author:** ![sandeep15mca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sandeep15mca/32/3178_2.png) [@sandeep15mca](https://discuss.elastic.co/u/sandeep15mca)\ **Post date:** [July 7, 2011, 11:06am UTC](https://discuss.elastic.co/t/filter-facets-and-filtered-query/4169/9 "2011-07-07T11:06:47Z")
Sir! I am new in Elasticsearch I know little bit anout it but I don't know how to configure it to my system anybody can help me. stepwise please write because after reading doc I couldn't understand please help me ASAP. ``` thanks. ``` On Thu, Jul 7, 2011 at 2:47 PM, Clinton Gormley [clinton@iannounce.co.uk](mailto:clinton@iannounce.co.uk)wrote: > Hi Cedric > > On Thu, 2011-07-07 at 00:41 -0700, melix wrote: > > > I'm also confused about this filter/facet combination. I can see why you > > would want to add specific filters to facets, but I don't understand why > > the > > global filter is not used to compute facets. For example, if you do : > > { "query": ..., > > "filter": ..., > > "facets": ... } > > > > Then the facet counts are returned without taking into account the > > filter. > > This seems illogic to me, as the documentation states that the facets are > > by > > default bound to the current query. Either the documentation should be > > more > > precise, telling that the filter is ignored, or there should be an option > > to > > make the filter active in facets too (which I think would be better for > > both > > backward compatibility and usability). > > "the facets are by default bound to the current query" > > This means the query that is specified in the "query" parameter. > > Normally, if you want a filtered query, you do: > > { > "query": { > "filtered": { > "query": { find foo }, > "filter": { filter by bar } > }, > "facets": { ... } # foo filtered by bar > } > > Facets will be bound by the filter specified in that query. > > The TOP LEVEL filter parameter was added so that you can get your facet > results on all documents matching the query { find foo }, then later > filter your query results with { filter by bar }, which would look like: > > { > "query": { find foo }, > "filter": { filter by bar }, > "facets": { ... } # foo, not filtered > } > > clint ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [July 7, 2011, 2:55pm UTC](https://discuss.elastic.co/t/filter-facets-and-filtered-query/4169/10 "2011-07-07T14:55:06Z")
Heya, The reasoning behind the "global" filter not affecting facets is facet navigation. When you start to filter your search results by facets ("navigate them"), some facets should still compute based on the original user entered query, and not the filtering done based on the facet selection. -shay.banon On Thursday, July 7, 2011 at 12:58 PM, Cédric Champeau wrote: > Sure, but I think you also miss some background about how my queries are > generated. In fact, I have a legacy application which has its own query > format. I am able to map this query format to various search engines, > including Lucene, MG4J and now I am working on Elasticsearch. This > abstract query format is quite old and while I am able to map it to > various search engines, it's logic is sometimes slightly different from > the underlying search engine. This abstract model does have a notion of > filter on a query, and I could use it, but there is another problem : > > The second point is about the query format. My abstract query model is > an object model. The input format is parsed into this abstract model. I > had XML until now, and I'm adding a JSON format now. I posted a message > yesterday regarding the ability to use the internal JSON parsers from ES > to be able to reuse the Elasticsearch syntax at some points. > Unfortunately, this is not possible unless using the "extraSource" > parameter of a query. Having those two points in mind, I have a choice > to make : > > - add the ability to use the Elasticsearch JSON syntax of a query > filter inside my own JSON query format, but it would require me to write > parsers for the various filters ES supports > - add an "extraSource" section into my custom JSON format where the > user may add a filter. This is the solution I managed to implement. > While it has the restriction of not being able to add a filter on every > subquery, it is perfectly acceptable. However, the problem with this > solution is that the user must use a boolean AND query in order to > emulate a filter at the query level to have the filters "used" by the > facets. Indeed, this is not a filter and suffers performance penalties. > > Having a flag allowing the "global filter" to be used by facets would > solve this issue for me. The alternative, reimplementing json parsers > where I wanted the very same format as Elasticsearch to be supported, > looks unproductive. > > I hope this makes it clearer why I think this flag would be useful 😃 > > Le 07/07/2011 11:38, Clinton Gormley a écrit : > > > On Thu, 2011-07-07 at 11:29 +0200, Cédric Champeau wrote: > > > > > I understand, but in a facetted search, I would expect everything to be > > > refined, so the facet counts should IMHO change whenever I add a filter. > > > I still think an option to make have the global filter used by the > > > facets would be nice 🙂 > > > Cedric, I think you missed my point. The top-level filter is there ONLY > > > for situations where you specifically DON'T want the facets to be bound > > > by your filter. > > > > So if you use the filtered query, then it will do exactly what you want. > > > > clint ---
**Author:** ![Meghan\_Mahoney](https://avatars.discourse-cdn.com/v4/letter/m/a6a055/32.png) [@Meghan\_Mahoney](https://discuss.elastic.co/u/Meghan_Mahoney)\ **Post date:** [November 25, 2013, 4:31pm UTC](https://discuss.elastic.co/t/filter-facets-and-filtered-query/4169/11 "2013-11-25T16:31:29Z")
Here's another example of using a filtered query with a facet: When I first started using facet queries they were bringing down the cluster after looking around in the documentation I discovered that the below query I was trying actually put all messages from the index with my filter field into the filter cache rather than just the messages I'm interested in. { "query": { "query\_string": { "query": "short\_message:(650 OR 500) AND @timestamp:[2013-11-04T21:00:00 TO 2013-11-04T21:00:01]" } }, "size": 1, "facets": { "facet": { "terms": { "field": "full\_message\_na", "size": 500 } } } } This query gives the same results but only does the facet on the results of the filtered query. And did not bring down the cluster. { "query": { "filtered": { "query": { "match\_all": {} }, "filter": { "and": [ { "range": { "@timestamp": { "from": "2013-11-04T21:00:00", "to": "2013-11-04T21:00:05" } } }, { "terms": { "short\_message": [ "500", "650" ] } } ] } } }, "facets": { "my\_facet": { "terms": { "field": "full\_message\_na", "size": 10 } } }, "size": 1 } The downside to the above query is that it does not use the query string format for the filter so in order to use it i will have to change the input people are giving my script. For now I'm still using the old query and have just limited the facet cache to 10% of heap. Therefore it never fills up. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:04am UTC](https://discuss.elastic.co/t/filter-facets-and-filtered-query/4169/12 "2017-07-06T02:04:57Z")