# When searching for 'Boss' with fuzziness, get higher score for 'Bose' than 'Boss'. ? How Comes !?!? **URL:** \ **Category:** Elasticsearch\ **Created:** [January 15, 2015, 3:06pm UTC](https://discuss.elastic.co/t/when-searching-for-boss-with-fuzziness-get-higher-score-for-bose-than-boss-how-comes/21662 "2015-01-15T15:06:06Z")\ **Posts on this page:** 1\ **Showing post:** 8
**Author:** ![Mark\_Harwood\_2](https://avatars.discourse-cdn.com/v4/letter/m/c77e96/32.png) [@Mark\_Harwood\_2](https://discuss.elastic.co/u/Mark_Harwood_2)\ **Post date:** [January 20, 2015, 9:20am UTC](https://discuss.elastic.co/t/when-searching-for-boss-with-fuzziness-get-higher-score-for-bose-than-boss-how-comes/21662/8 "2015-01-20T09:20:58Z")
🙂 It's been nearly 10 years but I took a quick look at the code and the IDF balancing stuff is still in there. Testing queries against a large index of cars Lucene's standard fuzzy query on ford~ still has top matches that aren't Fords. FLT works fine. On Tuesday, January 20, 2015 at 9:11:56 AM UTC, Itamar Syn-Hershko wrote: > Famous last words 🙂 > > -- > > Itamar Syn-Hershko > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) > Freelance Developer & Consultant > [Lucene.NET](http://Lucene.NET) committer and PMC member > > On Tue, Jan 20, 2015 at 11:11 AM, Mark Harwood \< > [mark.h...@elasticsearch.com](mailto:mark.h...@elasticsearch.com) \\> wrote: > > > > it doesn't seem like this would address the IDF > > > > Trust me, I wrote it. > > > > On Tuesday, January 20, 2015 at 12:16:44 AM UTC, [kasper...@yahoo.com](mailto:kasper...@yahoo.com) > > wrote: > > > > > Thanks Mark. Sounds like this issue affects a lot of people. > > > > > > I looked at your suggestion about FLT, and the ignore\_tf parameter > > > should help, however unless I'm missing something, it doesn't seem like > > > this would address the IDF, and results could be biased. But I will > > > experiment. > > > > > > Ultimately I think what my particular use case requires is a scorer that > > > only uses edit distance (when querying with fuzziness) and field boosts, > > > but no TF / IDF. > > > > > > On Monday, January 19, 2015 at 3:15:47 PM UTC-8, Mark Harwood wrote: > > > > > > > This issue rounds up a bunch of related issues that have been raised > > > > previously: [Wrap stacked tokens in `match` query in a BlendedTerms query for better scoring · Issue #9103 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/9103) > > > > > > > > For now try FuzzyLikeThis ([http://www.elasticsearch.org/](http://www.elasticsearch.org/) > > > > guide/en/elasticsearch/reference/current/query-dsl- > > > > flt-query.html#query-dsl-flt-query ) > > > > It blends More Like This and fuzzy functionality but includes the > > > > adjustments to IDF that I think make more sense than the other > > > > implementations with their bias towards rewarding scarcity. > > > > > > > > On Monday, January 19, 2015 at 6:48:49 PM UTC, [kasper...@yahoo.com](mailto:kasper...@yahoo.com) > > > > wrote: > > > > > > > > > I have the same problem, where some results with higher edit distance > > > > > are ranked higher than other results that are closer in terms of edit > > > > > distance. > > > > > > > > > > I suspect it does have to do with document frequency, as you think > > > > > Adrien. In my case I want to ignore document frequency completely. Any > > > > > suggestion to achieve this? > > > > > > > > > > I'm a taker of any solution as this looks like a show stopper for us, > > > > > so even a workaround would help. > > > > > > > > > > I can try to create this other rewrite method you mentioned if you > > > > > could point me in the right direction. > > > > > > > > > > Thanks > > > > > > > > > > On Thursday, January 15, 2015 at 7:44:57 AM UTC-8, Adrien Grand wrote: > > > > > > > > > > > This is because the score takes two factors into account: the > > > > > > document frequency and the edit distance. Quite likely in your case, even > > > > > > though Boss is closer than Bose, Bose has a much lower document frequency > > > > > > which helped it eventually get a better score. I guess we should have > > > > > > another rewrite method that would not take freqs into account (or somehow > > > > > > merge them) to avoid that issue. > > > > > > > > > > > > On Thu, Jan 15, 2015 at 4:06 PM, Eylon Steiner [eylon....@gmail.com](mailto:eylon....@gmail.com) > > > > > > wrote: > > > > > > > > > > > > > Any ideas? > > > > > > > > > > > > > > -- > > > > > > > You received this message because you are subscribed to the Google > > > > > > > Groups "elasticsearch" group. > > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > > > To view this discussion on the web visit > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/52e09e54-](https://groups.google.com/d/msgid/elasticsearch/52e09e54-) > > > > > > > 90b6-4014-8454-34e3db5756e5%[40googlegroups.com](http://40googlegroups.com) > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > > > . > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > > > -- > > > > > > Adrien Grand > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google Groups > > > > > "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send an > > > > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > > > To view this discussion on the web visit > > > > > [https://groups.google.com/d/msgid/elasticsearch/9523b3d5-ffea-4760-9782-69167b9807ed%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/9523b3d5-ffea-4760-9782-69167b9807ed%40googlegroups.com) > > > > > [https://groups.google.com/d/msgid/elasticsearch/9523b3d5-ffea-4760-9782-69167b9807ed%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/9523b3d5-ffea-4760-9782-69167b9807ed%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > . > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/84f178ba-d150-4b7d-9a54-b419bc962499%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/84f178ba-d150-4b7d-9a54-b419bc962499%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). --- _[View the full topic](https://discuss.elastic.co/t/when-searching-for-boss-with-fuzziness-get-higher-score-for-bose-than-boss-how-comes/21662)._