# Stopwords and minimum\_should\_match in multi-field query\_string **URL:** \ **Category:** Elasticsearch\ **Created:** [April 2, 2012, 8:07pm UTC](https://discuss.elastic.co/t/stopwords-and-minimum-should-match-in-multi-field-query-string/7203 "2012-04-02T20:07:00Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Matthew\_A\_Brown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/matthew_a_brown/32/2699_2.png) [@Matthew\_A\_Brown](https://discuss.elastic.co/u/Matthew_A_Brown)\ **Post date:** [April 2, 2012, 8:07pm UTC](https://discuss.elastic.co/t/stopwords-and-minimum-should-match-in-multi-field-query-string/7203/1 "2012-04-02T20:07:00Z")
I've run into an interesting conundrum. I don't think this is a bug, but I'm also not sure how to get the behavior I want, so I was hoping someone might have a brilliant idea. Let's say I've got a type "books", with a "title" and "author" field. Let's further say that I'm indexing "title" with a stopwords filter, but "author" has no stopwords filter. The following document is the only one in the index: {"title":"The Great Gatsby", "author":"F. Scott Fitzgerald"} Now let's say I want to perform the following search: {"query": {"query\_string": {"query": "the great gatsby", "fields": ["title", "author"], "minimum\_should\_match": 3}}} This search won't return any results. I believe this is because the minimum\_should\_match is 3, and the stopwords filter is dropping "the". So, only two tokens match, but since the original input string had 3 tokens, it's still looking for a 3-token match. However, the behavior is different if I only search the "title" field: {"query": {"query\_string": {"query": "the great gatsby", "fields": ["title"], "minimum\_should\_match": 3}}} In this case, I do get a result back, presumably because the only analyzer in play is only generating two search tokens, so that becomes the ceiling for the purposes of the minimum\_should\_match. I'm not sure how that ceiling is calculated by the DisMax parser -- is it the max # of terms generated by any component of the disjunction? I don't suppose it would be possible for the minimum\_should\_match to be "local" to each component? This could easily be getting pretty deep into Lucene internals. Anyway, just wondering if anyone has any fantastic insight : ) Thanks! Mat ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [April 3, 2012, 2:30pm UTC](https://discuss.elastic.co/t/stopwords-and-minimum-should-match-in-multi-field-query-string/7203/2 "2012-04-03T14:30:54Z")
Yea, minimum should match is a bit meaningless with multi field querying, because the boolean query generated is one of several dis max queries broken down to queries on each "field" parsed by the query parser. On Mon, Apr 2, 2012 at 11:07 PM, Matthew A. Brown [mat.a.brown@gmail.com](mailto:mat.a.brown@gmail.com)wrote: > I've run into an interesting conundrum. I don't think this is a bug, > but I'm also not sure how to get the behavior I want, so I was hoping > someone might have a brilliant idea. > > Let's say I've got a type "books", with a "title" and "author" field. > Let's further say that I'm indexing "title" with a stopwords filter, > but "author" has no stopwords filter. > > The following document is the only one in the index: > > {"title":"The Great Gatsby", "author":"F. Scott Fitzgerald"} > > Now let's say I want to perform the following search: > > {"query": {"query\_string": {"query": "the great gatsby", "fields": > ["title", "author"], "minimum\_should\_match": 3}}} > > This search won't return any results. I believe this is because the > minimum\_should\_match is 3, and the stopwords filter is dropping "the". > So, only two tokens match, but since the original input string had 3 > tokens, it's still looking for a 3-token match. > > However, the behavior is different if I only search the "title" field: > > {"query": {"query\_string": {"query": "the great gatsby", "fields": > ["title"], "minimum\_should\_match": 3}}} > > In this case, I do get a result back, presumably because the only > analyzer in play is only generating two search tokens, so that becomes > the ceiling for the purposes of the minimum\_should\_match. > > I'm not sure how that ceiling is calculated by the DisMax parser -- is > it the max # of terms generated by any component of the disjunction? I > don't suppose it would be possible for the minimum\_should\_match to be > "local" to each component? This could easily be getting pretty deep > into Lucene internals. > > Anyway, just wondering if anyone has any fantastic insight : ) > > Thanks! > > Mat ---
**Author:** ![Matthew\_A\_Brown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/matthew_a_brown/32/2699_2.png) [@Matthew\_A\_Brown](https://discuss.elastic.co/u/Matthew_A_Brown)\ **Post date:** [April 3, 2012, 2:34pm UTC](https://discuss.elastic.co/t/stopwords-and-minimum-should-match-in-multi-field-query-string/7203/3 "2012-04-03T14:34:38Z")
Thanks, Shay! Any suggestions on how to get the behavior I'm looking for? On Tue, Apr 3, 2012 at 10:30, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > Yea, minimum should match is a bit meaningless with multi field querying, > because the boolean query generated is one of several dis max queries broken > down to queries on each "field" parsed by the query parser. > > On Mon, Apr 2, 2012 at 11:07 PM, Matthew A. Brown [mat.a.brown@gmail.com](mailto:mat.a.brown@gmail.com) > wrote: > > > I've run into an interesting conundrum. I don't think this is a bug, > > but I'm also not sure how to get the behavior I want, so I was hoping > > someone might have a brilliant idea. > > > > Let's say I've got a type "books", with a "title" and "author" field. > > Let's further say that I'm indexing "title" with a stopwords filter, > > but "author" has no stopwords filter. > > > > The following document is the only one in the index: > > > > {"title":"The Great Gatsby", "author":"F. Scott Fitzgerald"} > > > > Now let's say I want to perform the following search: > > > > {"query": {"query\_string": {"query": "the great gatsby", "fields": > > ["title", "author"], "minimum\_should\_match": 3}}} > > > > This search won't return any results. I believe this is because the > > minimum\_should\_match is 3, and the stopwords filter is dropping "the". > > So, only two tokens match, but since the original input string had 3 > > tokens, it's still looking for a 3-token match. > > > > However, the behavior is different if I only search the "title" field: > > > > {"query": {"query\_string": {"query": "the great gatsby", "fields": > > ["title"], "minimum\_should\_match": 3}}} > > > > In this case, I do get a result back, presumably because the only > > analyzer in play is only generating two search tokens, so that becomes > > the ceiling for the purposes of the minimum\_should\_match. > > > > I'm not sure how that ceiling is calculated by the DisMax parser -- is > > it the max # of terms generated by any component of the disjunction? I > > don't suppose it would be possible for the minimum\_should\_match to be > > "local" to each component? This could easily be getting pretty deep > > into Lucene internals. > > > > Anyway, just wondering if anyone has any fantastic insight : ) > > > > Thanks! > > > > Mat ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [April 4, 2012, 1:49pm UTC](https://discuss.elastic.co/t/stopwords-and-minimum-should-match-in-multi-field-query-string/7203/4 "2012-04-04T13:49:01Z")
The only thing that I can think is using dismax explicitly, and have several query\_string queries in it. It won't behave the same as specifying fields in the query\_string (the dismax will be on the whole parsed query), but it might make sense. On Tue, Apr 3, 2012 at 5:34 PM, Matthew A. Brown [mat.a.brown@gmail.com](mailto:mat.a.brown@gmail.com)wrote: > Thanks, Shay! Any suggestions on how to get the behavior I'm looking for? > > On Tue, Apr 3, 2012 at 10:30, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > > > Yea, minimum should match is a bit meaningless with multi field querying, > > because the boolean query generated is one of several dis max queries > > broken > > down to queries on each "field" parsed by the query parser. > > > > On Mon, Apr 2, 2012 at 11:07 PM, Matthew A. Brown \<[mat.a.brown@gmail.com](mailto:mat.a.brown@gmail.com) > > > > wrote: > > > > > I've run into an interesting conundrum. I don't think this is a bug, > > > but I'm also not sure how to get the behavior I want, so I was hoping > > > someone might have a brilliant idea. > > > > > > Let's say I've got a type "books", with a "title" and "author" field. > > > Let's further say that I'm indexing "title" with a stopwords filter, > > > but "author" has no stopwords filter. > > > > > > The following document is the only one in the index: > > > > > > {"title":"The Great Gatsby", "author":"F. Scott Fitzgerald"} > > > > > > Now let's say I want to perform the following search: > > > > > > {"query": {"query\_string": {"query": "the great gatsby", "fields": > > > ["title", "author"], "minimum\_should\_match": 3}}} > > > > > > This search won't return any results. I believe this is because the > > > minimum\_should\_match is 3, and the stopwords filter is dropping "the". > > > So, only two tokens match, but since the original input string had 3 > > > tokens, it's still looking for a 3-token match. > > > > > > However, the behavior is different if I only search the "title" field: > > > > > > {"query": {"query\_string": {"query": "the great gatsby", "fields": > > > ["title"], "minimum\_should\_match": 3}}} > > > > > > In this case, I do get a result back, presumably because the only > > > analyzer in play is only generating two search tokens, so that becomes > > > the ceiling for the purposes of the minimum\_should\_match. > > > > > > I'm not sure how that ceiling is calculated by the DisMax parser -- is > > > it the max # of terms generated by any component of the disjunction? I > > > don't suppose it would be possible for the minimum\_should\_match to be > > > "local" to each component? This could easily be getting pretty deep > > > into Lucene internals. > > > > > > Anyway, just wondering if anyone has any fantastic insight : ) > > > > > > Thanks! > > > > > > Mat ---
**Author:** ![Matthew\_A\_Brown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/matthew_a_brown/32/2699_2.png) [@Matthew\_A\_Brown](https://discuss.elastic.co/u/Matthew_A_Brown)\ **Post date:** [April 4, 2012, 1:50pm UTC](https://discuss.elastic.co/t/stopwords-and-minimum-should-match-in-multi-field-query-string/7203/5 "2012-04-04T13:50:23Z")
Thanks, Shay. I did try that but as you mentioned, the behavior isn't quite equivalent. I'll have to do some more thinking on it... On Wed, Apr 4, 2012 at 09:49, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > The only thing that I can think is using dismax explicitly, and have several > query\_string queries in it. It won't behave the same as specifying fields in > the query\_string (the dismax will be on the whole parsed query), but it > might make sense. > > On Tue, Apr 3, 2012 at 5:34 PM, Matthew A. Brown [mat.a.brown@gmail.com](mailto:mat.a.brown@gmail.com) > wrote: > > > Thanks, Shay! Any suggestions on how to get the behavior I'm looking for? > > > > On Tue, Apr 3, 2012 at 10:30, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > > > > > Yea, minimum should match is a bit meaningless with multi field > > > querying, > > > because the boolean query generated is one of several dis max queries > > > broken > > > down to queries on each "field" parsed by the query parser. > > > > > > On Mon, Apr 2, 2012 at 11:07 PM, Matthew A. Brown > > > [mat.a.brown@gmail.com](mailto:mat.a.brown@gmail.com) > > > wrote: > > > > > > > I've run into an interesting conundrum. I don't think this is a bug, > > > > but I'm also not sure how to get the behavior I want, so I was hoping > > > > someone might have a brilliant idea. > > > > > > > > Let's say I've got a type "books", with a "title" and "author" field. > > > > Let's further say that I'm indexing "title" with a stopwords filter, > > > > but "author" has no stopwords filter. > > > > > > > > The following document is the only one in the index: > > > > > > > > {"title":"The Great Gatsby", "author":"F. Scott Fitzgerald"} > > > > > > > > Now let's say I want to perform the following search: > > > > > > > > {"query": {"query\_string": {"query": "the great gatsby", "fields": > > > > ["title", "author"], "minimum\_should\_match": 3}}} > > > > > > > > This search won't return any results. I believe this is because the > > > > minimum\_should\_match is 3, and the stopwords filter is dropping "the". > > > > So, only two tokens match, but since the original input string had 3 > > > > tokens, it's still looking for a 3-token match. > > > > > > > > However, the behavior is different if I only search the "title" field: > > > > > > > > {"query": {"query\_string": {"query": "the great gatsby", "fields": > > > > ["title"], "minimum\_should\_match": 3}}} > > > > > > > > In this case, I do get a result back, presumably because the only > > > > analyzer in play is only generating two search tokens, so that becomes > > > > the ceiling for the purposes of the minimum\_should\_match. > > > > > > > > I'm not sure how that ceiling is calculated by the DisMax parser -- is > > > > it the max # of terms generated by any component of the disjunction? I > > > > don't suppose it would be possible for the minimum\_should\_match to be > > > > "local" to each component? This could easily be getting pretty deep > > > > into Lucene internals. > > > > > > > > Anyway, just wondering if anyone has any fantastic insight : ) > > > > > > > > Thanks! > > > > > > > > Mat ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:33am UTC](https://discuss.elastic.co/t/stopwords-and-minimum-should-match-in-multi-field-query-string/7203/6 "2017-07-06T03:33:41Z")