# Apply query after filter **URL:** \ **Category:** Elasticsearch\ **Created:** [December 16, 2011, 10:01am UTC](https://discuss.elastic.co/t/apply-query-after-filter/6176 "2011-12-16T10:01:31Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![George\_Sakkis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/george_sakkis/32/2801_2.png) [@George\_Sakkis](https://discuss.elastic.co/u/George_Sakkis)\ **Post date:** [December 16, 2011, 10:01am UTC](https://discuss.elastic.co/t/apply-query-after-filter/6176/1 "2011-12-16T10:01:31Z")
Hi all, according to the docs a FilteredQuery is a query that applies a filter to the results of another query. Is there a way to reverse the order so that the subquery runs on the results of the filter? At least in my case this would have big performance impact as the filter limits the candidate set to a few hundreds docs down from millions. Thanks, George ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [December 16, 2011, 5:05pm UTC](https://discuss.elastic.co/t/apply-query-after-filter/6176/2 "2011-12-16T17:05:34Z")
The actual execution is interleaving, so you are good. On Fri, Dec 16, 2011 at 12:01 PM, George Sakkis [george.sakkis@gmail.com](mailto:george.sakkis@gmail.com)wrote: > Hi all, > > according to the docs a FilteredQuery is a query that applies a filter > to the results of another query. Is there a way to reverse the order > so that the subquery runs on the results of the filter? At least in my > case this would have big performance impact as the filter limits the > candidate set to a few hundreds docs down from millions. > > Thanks, > George ---
**Author:** ![George\_Sakkis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/george_sakkis/32/2801_2.png) [@George\_Sakkis](https://discuss.elastic.co/u/George_Sakkis)\ **Post date:** [December 19, 2011, 9:22am UTC](https://discuss.elastic.co/t/apply-query-after-filter/6176/3 "2011-12-19T09:22:02Z")
Unfortunately this doesn't seem to be enough, at least for broad prefix queries there is a two orders of magnitude difference in my index. Here's my original query, it takes ~650ms for 25 hits: { "query": { "filtered": { "filter": { "terms": { "user\_id": [ 291773 ] } }, "query": { "prefix": { "\_all": { "prefix": "m" } } } } } } Replacing the prefix query with a match\_all returns 147 hits in only 5ms! { "query": { "filtered": { "filter": { "terms": { "app\_id": [ 291773 ] } }, "query": { "match\_all": {} } } } } The workaround I resorted to for now is to use a PrefixFilter instead of PrefixQuery and rely on caching but ideally I would like to use a query and specify/override the execution order. Any way to force the query to run on the result set of the filter? On Dec 16, 6:05 pm, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > The actual execution is interleaving, so you are good. > > On Fri, Dec 16, 2011 at 12:01 PM, George Sakkis [george.sak...@gmail.com](mailto:george.sak...@gmail.com)wrote: > > > Hi all, > > > according to the docs a FilteredQuery is a query that applies a filter > > to the results of another query. Is there a way to reverse the order > > so that the subquery runs on the results of the filter? At least in my > > case this would have big performance impact as the filter limits the > > candidate set to a few hundreds docs down from millions. > > > Thanks, > > George ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [December 19, 2011, 9:58am UTC](https://discuss.elastic.co/t/apply-query-after-filter/6176/4 "2011-12-19T09:58:31Z")
Hi George On Mon, 2011-12-19 at 01:22 -0800, George Sakkis wrote: > Unfortunately this doesn't seem to be enough, at least for broad > prefix queries there is a two orders of magnitude difference in my > index. Here's my original query, it takes ~650ms for 25 hits: The problem with your query is not filter vs query, it is the use of the prefix query (or filter for that matter). It is not an efficient search. It has to: - load all terms - find all terms beginning with 'm' - limit those to the 1024 most relevant 'm' terms - do 1024 searches prefix clauses are ok when you have few terms, but not for general use. What you want to do is to prepare your data properly, ie index appropriate fields with the 'edgeNGram' token filter, which will produce terms like: 'm','ma','mar','mary' Then a search for eg 'ma' will be quick. Note: your search\_analayzer should NOT include the edgeNGram filter, otherwise it'll search for 'm','ma' etc have a look at this previous reply for more detail: [http://elasticsearch-users.115913.n3.nabble.com/help-needed-with-the-query-tt3177477.html#a3178856](http://elasticsearch-users.115913.n3.nabble.com/help-needed-with-the-query-tt3177477.html#a3178856) clint ---
**Author:** ![George\_Sakkis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/george_sakkis/32/2801_2.png) [@George\_Sakkis](https://discuss.elastic.co/u/George_Sakkis)\ **Post date:** [December 19, 2011, 1:01pm UTC](https://discuss.elastic.co/t/apply-query-after-filter/6176/5 "2011-12-19T13:01:58Z")
Hi Clinton, On Dec 19, 10:58 am, Clinton Gormley [cl...@traveljury.com](mailto:cl...@traveljury.com) wrote: > Hi George > > On Mon, 2011-12-19 at 01:22 -0800, George Sakkis wrote: > > > Unfortunately this doesn't seem to be enough, at least for broad > > prefix queries there is a two orders of magnitude difference in my > > index. Here's my original query, it takes ~650ms for 25 hits: > > The problem with your query is not filter vs query, it is the use of the > prefix query (or filter for that matter). > > It is not an efficient search. It has to: > > - load all terms > - find all terms beginning with 'm' > - limit those to the 1024 most relevant 'm' terms > - do 1024 searches > > prefix clauses are ok when you have few terms, but not for general use. The thing is it's not general use in my case; ideally the prefix search would be applied to only the 147 documents that match the other filter, if only I could specify the execution order. George ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [December 20, 2011, 3:01pm UTC](https://discuss.elastic.co/t/apply-query-after-filter/6176/6 "2011-12-20T15:01:27Z")
Thats not really how prefix / wildcard queries work, in Lucene, there is a rewrite phase for queries that happens before the actual query execution, and in this phase prefix/wildcard can be expensive. On Mon, Dec 19, 2011 at 3:01 PM, George Sakkis [george.sakkis@gmail.com](mailto:george.sakkis@gmail.com)wrote: > Hi Clinton, > > On Dec 19, 10:58 am, Clinton Gormley [cl...@traveljury.com](mailto:cl...@traveljury.com) wrote: > > > Hi George > > > > On Mon, 2011-12-19 at 01:22 -0800, George Sakkis wrote: > > > > > Unfortunately this doesn't seem to be enough, at least for broad > > > prefix queries there is a two orders of magnitude difference in my > > > index. Here's my original query, it takes ~650ms for 25 hits: > > > > The problem with your query is not filter vs query, it is the use of the > > prefix query (or filter for that matter). > > > > It is not an efficient search. It has to: > > > > - load all terms > > - find all terms beginning with 'm' > > - limit those to the 1024 most relevant 'm' terms > > - do 1024 searches > > > > prefix clauses are ok when you have few terms, but not for general use. > > The thing is it's not general use in my case; ideally the prefix > search would be applied to only the 147 documents that match the other > filter, if only I could specify the execution order. > > George ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:44am UTC](https://discuss.elastic.co/t/apply-query-after-filter/6176/7 "2017-07-06T03:44:55Z")