# How to efficiently use stemmer to improve search results? **URL:** \ **Category:** Elasticsearch\ **Created:** [June 10, 2013, 4:00pm UTC](https://discuss.elastic.co/t/how-to-efficiently-use-stemmer-to-improve-search-results/12344 "2013-06-10T16:00:07Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Damien\_Alexandre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/damien_alexandre/32/21566_2.png) [@Damien\_Alexandre](https://discuss.elastic.co/u/Damien_Alexandre)\ **Post date:** [June 10, 2013, 4:00pm UTC](https://discuss.elastic.co/t/how-to-efficiently-use-stemmer-to-improve-search-results/12344/1 "2013-06-10T16:00:07Z")
Hi there, look at my gist! [https://gist.github.com/damienalexandre/5749824](https://gist.github.com/damienalexandre/5749824) I use a "francais" analyzer, doing some stuffs: - lowercase - asciifolding - elision - stemmer\_fr It use the built-in minimal\_french stemmer, so _Queens_ become _queen_. My question is, if I have a "_Queens of the stone age_" doc and a "_Queen_" doc, when I search "_Queens_", I want both documents but "_Queens of the stone age_" should score higher as it contains the whole word. If you run my gist you will see that both searches return the exact same result, that perfectly normal as my stored token are all "_queen_". But I'm looking for a way to use the stemmer only as a "second chance" search. I prefer my search results when I disable it, but I also want to find documents titled "_forgives_" when I search "_forgive_". Is there a way to do that with ElasticSearch ? Thank a lot 🙂 Damien -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![roytmana](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/roytmana/32/44855_2.png) [@roytmana](https://discuss.elastic.co/u/roytmana)\ **Post date:** [June 10, 2013, 5:55pm UTC](https://discuss.elastic.co/t/how-to-efficiently-use-stemmer-to-improve-search-results/12344/2 "2013-06-10T17:55:34Z")
Use multi field to index your field twice once with and once without stemming when searching search against both fields boosting the one without stem. you will probably end up having to use search\_string queries of you need to AND words in your query to search across both fields on any permutation of stemmed/unstemmed words. I also use technique of putting both stemmed and unstemmed tokens into the same field as opposed to two but it has some pitfalls -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Damien\_Alexandre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/damien_alexandre/32/21566_2.png) [@Damien\_Alexandre](https://discuss.elastic.co/u/Damien_Alexandre)\ **Post date:** [June 11, 2013, 8:10am UTC](https://discuss.elastic.co/t/how-to-efficiently-use-stemmer-to-improve-search-results/12344/3 "2013-06-11T08:10:03Z")
I would like to query my documents against the _\_all_ field, that's why in my example you can see that the "_francais_" analyzer is run on "_\_all_". I have tried the multi\_field but I can't use it against the \_all field, documentation ( [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/multi-field-type/)) say this: The include\_in\_all setting on the “default” field allows to control if the value of the field should be included in the \_all field. Note, the value of the field is copied to \_all, not the tokens. So, it only makes sense to copy the field value once. Because of this, the include\_in\_all setting on all non-default fields is automatically set to false and can’t be changed. Here is my test file: [pony2.sh · GitHub](https://gist.github.com/damienalexandre/5755194) It could work as exepted if the stemmer was a tokenizer keeping the original word, so _Queens_ became [_Queen, Queens_] and not just [_Queen_]. Is that possible? Thank for your time 🙂 Damien On 10 June 2013 19:55, AlexR [roytmana@gmail.com](mailto:roytmana@gmail.com) wrote: > Use multi field to index your field twice once with and once without > stemming when searching search against both fields boosting the one without > stem. you will probably end up having to use search\_string queries of you > need to AND words in your query to search across both fields on any > permutation of stemmed/unstemmed words. > I also use technique of putting both stemmed and unstemmed tokens into the > same field as opposed to two but it has some pitfalls > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/KZaPM5KghYU/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/KZaPM5KghYU/unsubscribe?hl=en-US) > . > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Damien\_Alexandre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/damien_alexandre/32/21566_2.png) [@Damien\_Alexandre](https://discuss.elastic.co/u/Damien_Alexandre)\ **Post date:** [June 11, 2013, 10:14am UTC](https://discuss.elastic.co/t/how-to-efficiently-use-stemmer-to-improve-search-results/12344/4 "2013-06-11T10:14:51Z")
I may have found a proper way to do it, feel free to tell me if that's seems good to you too! I remove stemming from the _\_all_ field, but I apply it on the _name_ field. And I query like this: curl -XGET '[http://localhost:9200/pony\_index\_tmp/\_search?pretty=true](http://localhost:9200/pony_index_tmp/_search?pretty=true)' -d ' {"query": { "multi\_match" : { "query" : "queens", "fields" : ["\_all^2", "name"] } } }' And the results are better: 1: Gypsy Queens 2: Queens of the stone age 3: Queen I have no idea of how my "_name_" stemmed field is inserted into the \_all one, does it get stemmed?! That's not clear at all for me. Thank you, happy coding! On 11 June 2013 10:10, Damien Alexandre [dalexandre@jolicode.com](mailto:dalexandre@jolicode.com) wrote: > I would like to query my documents against the _\_all_ field, > that's why in my example you can see that the "_francais_" analyzer is > run on "_\_all_". > > I have tried the multi\_field but I can't use it against the \_all field, > documentation ( > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/multi-field-type/)) > say this: > > The include\_in\_all setting on the “default” field allows to control if the > value of the field should be included in the \_all field. Note, the value of > the field is copied to \_all, not the tokens. So, it only makes sense to > copy the field value once. Because of this, the include\_in\_all setting on > all non-default fields is automatically set to false and can’t be changed. > > Here is my test file: [pony2.sh · GitHub](https://gist.github.com/damienalexandre/5755194) > > It could work as exepted if the stemmer was a tokenizer keeping the > original word, > so _Queens_ became [_Queen, Queens_] and not just [_Queen_]. Is that > possible? > > Thank for your time 🙂 > > Damien > > On 10 June 2013 19:55, AlexR [roytmana@gmail.com](mailto:roytmana@gmail.com) wrote: > > > Use multi field to index your field twice once with and once without > > stemming when searching search against both fields boosting the one without > > stem. you will probably end up having to use search\_string queries of you > > need to AND words in your query to search across both fields on any > > permutation of stemmed/unstemmed words. > > I also use technique of putting both stemmed and unstemmed tokens into > > the same field as opposed to two but it has some pitfalls > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/KZaPM5KghYU/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/KZaPM5KghYU/unsubscribe?hl=en-US) > > . > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![roytmana](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/roytmana/32/44855_2.png) [@roytmana](https://discuss.elastic.co/u/roytmana)\ **Post date:** [June 11, 2013, 3:14pm UTC](https://discuss.elastic.co/t/how-to-efficiently-use-stemmer-to-improve-search-results/12344/5 "2013-06-11T15:14:00Z")
Two things 1. Is name the only field you need to stem? Sounds unlikely given that you searching on \_all. So you will likely end up wit your cusom all-like multi field with stemmed properties which you can then combine with all 2. Bigger issue is that multimatch will not work if you need to AND your search phrase words as it will search for every word in every field so it will be like searching on name alone. So you would need to use search\_string -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:31am UTC](https://discuss.elastic.co/t/how-to-efficiently-use-stemmer-to-improve-search-results/12344/6 "2017-07-06T02:31:47Z")