# When searching for 'Boss' with fuzziness, get higher score for 'Bose' than 'Boss'. ? How Comes !?!? **URL:** \ **Category:** Elasticsearch\ **Created:** [January 15, 2015, 3:06pm UTC](https://discuss.elastic.co/t/when-searching-for-boss-with-fuzziness-get-higher-score-for-bose-than-boss-how-comes/21662 "2015-01-15T15:06:06Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Eylon\_Steiner](https://avatars.discourse-cdn.com/v4/letter/e/977dab/32.png) [@Eylon\_Steiner](https://discuss.elastic.co/u/Eylon_Steiner)\ **Post date:** [January 15, 2015, 3:06pm UTC](https://discuss.elastic.co/t/when-searching-for-boss-with-fuzziness-get-higher-score-for-bose-than-boss-how-comes/21662/1 "2015-01-15T15:06:06Z")
Any ideas? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jpountz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jpountz/32/45836_2.png) [@jpountz](https://discuss.elastic.co/u/jpountz)\ **Post date:** [January 15, 2015, 3:44pm UTC](https://discuss.elastic.co/t/when-searching-for-boss-with-fuzziness-get-higher-score-for-bose-than-boss-how-comes/21662/2 "2015-01-15T15:44:50Z")
This is because the score takes two factors into account: the document frequency and the edit distance. Quite likely in your case, even though Boss is closer than Bose, Bose has a much lower document frequency which helped it eventually get a better score. I guess we should have another rewrite method that would not take freqs into account (or somehow merge them) to avoid that issue. On Thu, Jan 15, 2015 at 4:06 PM, Eylon Steiner [eylon.steiner@gmail.com](mailto:eylon.steiner@gmail.com) wrote: > Any ideas? > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- Adrien Grand -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j7-7SbX\_CVizbC%3DwCf9jyNSfkn4zy-GEqEj0sdBZGkRrg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j7-7SbX_CVizbC%3DwCf9jyNSfkn4zy-GEqEj0sdBZGkRrg%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![ElasticSearch\_Users\_](https://avatars.discourse-cdn.com/v4/letter/e/b4bc9f/32.png) [@ElasticSearch\_Users\_](https://discuss.elastic.co/u/ElasticSearch_Users_)\ **Post date:** [January 19, 2015, 6:48pm UTC](https://discuss.elastic.co/t/when-searching-for-boss-with-fuzziness-get-higher-score-for-bose-than-boss-how-comes/21662/3 "2015-01-19T18:48:49Z")
I have the same problem, where some results with higher edit distance are ranked higher than other results that are closer in terms of edit distance. I suspect it does have to do with document frequency, as you think Adrien. In my case I want to ignore document frequency completely. Any suggestion to achieve this? I'm a taker of any solution as this looks like a show stopper for us, so even a workaround would help. I can try to create this other rewrite method you mentioned if you could point me in the right direction. Thanks On Thursday, January 15, 2015 at 7:44:57 AM UTC-8, Adrien Grand wrote: > This is because the score takes two factors into account: the document > frequency and the edit distance. Quite likely in your case, even though > Boss is closer than Bose, Bose has a much lower document frequency which > helped it eventually get a better score. I guess we should have another > rewrite method that would not take freqs into account (or somehow merge > them) to avoid that issue. > > On Thu, Jan 15, 2015 at 4:06 PM, Eylon Steiner \<[eylon....@gmail.com](mailto:eylon....@gmail.com) > \\> wrote: > > > Any ideas? > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > Adrien Grand -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/e92f143a-c4db-488d-9db4-7bedfaa14d2d%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e92f143a-c4db-488d-9db4-7bedfaa14d2d%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Mark\_Harwood\_2](https://avatars.discourse-cdn.com/v4/letter/m/c77e96/32.png) [@Mark\_Harwood\_2](https://discuss.elastic.co/u/Mark_Harwood_2)\ **Post date:** [January 19, 2015, 11:15pm UTC](https://discuss.elastic.co/t/when-searching-for-boss-with-fuzziness-get-higher-score-for-bose-than-boss-how-comes/21662/4 "2015-01-19T23:15:47Z")
This issue rounds up a bunch of related issues that have been raised previously: [Wrap stacked tokens in `match` query in a BlendedTerms query for better scoring · Issue #9103 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/9103) For now try FuzzyLikeThis ([Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-flt-query.html#query-dsl-flt-query) ) It blends More Like This and fuzzy functionality but includes the adjustments to IDF that I think make more sense than the other implementations with their bias towards rewarding scarcity. On Monday, January 19, 2015 at 6:48:49 PM UTC, [kasper...@yahoo.com](mailto:kasper...@yahoo.com) wrote: > I have the same problem, where some results with higher edit distance are > ranked higher than other results that are closer in terms of edit distance. > > I suspect it does have to do with document frequency, as you think Adrien. > In my case I want to ignore document frequency completely. Any suggestion > to achieve this? > > I'm a taker of any solution as this looks like a show stopper for us, so > even a workaround would help. > > I can try to create this other rewrite method you mentioned if you could > point me in the right direction. > > Thanks > > On Thursday, January 15, 2015 at 7:44:57 AM UTC-8, Adrien Grand wrote: > > > This is because the score takes two factors into account: the document > > frequency and the edit distance. Quite likely in your case, even though > > Boss is closer than Bose, Bose has a much lower document frequency which > > helped it eventually get a better score. I guess we should have another > > rewrite method that would not take freqs into account (or somehow merge > > them) to avoid that issue. > > > > On Thu, Jan 15, 2015 at 4:06 PM, Eylon Steiner [eylon....@gmail.com](mailto:eylon....@gmail.com) > > wrote: > > > > > Any ideas? > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > Adrien Grand -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/8328d71a-b2be-40aa-abdc-ebeddb9a713d%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/8328d71a-b2be-40aa-abdc-ebeddb9a713d%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![ElasticSearch\_Users\_](https://avatars.discourse-cdn.com/v4/letter/e/b4bc9f/32.png) [@ElasticSearch\_Users\_](https://discuss.elastic.co/u/ElasticSearch_Users_)\ **Post date:** [January 20, 2015, 12:16am UTC](https://discuss.elastic.co/t/when-searching-for-boss-with-fuzziness-get-higher-score-for-bose-than-boss-how-comes/21662/5 "2015-01-20T00:16:44Z")
Thanks Mark. Sounds like this issue affects a lot of people. I looked at your suggestion about FLT, and the ignore\_tf parameter should help, however unless I'm missing something, it doesn't seem like this would address the IDF, and results could be biased. But I will experiment. Ultimately I think what my particular use case requires is a scorer that only uses edit distance (when querying with fuzziness) and field boosts, but no TF / IDF. On Monday, January 19, 2015 at 3:15:47 PM UTC-8, Mark Harwood wrote: > This issue rounds up a bunch of related issues that have been raised > previously: [Wrap stacked tokens in `match` query in a BlendedTerms query for better scoring · Issue #9103 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/9103) > > For now try FuzzyLikeThis ( > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-flt-query.html#query-dsl-flt-query) > ) > It blends More Like This and fuzzy functionality but includes the > adjustments to IDF that I think make more sense than the other > implementations with their bias towards rewarding scarcity. > > On Monday, January 19, 2015 at 6:48:49 PM UTC, [kasper...@yahoo.com](mailto:kasper...@yahoo.com) wrote: > > > I have the same problem, where some results with higher edit distance are > > ranked higher than other results that are closer in terms of edit distance. > > > > I suspect it does have to do with document frequency, as you think > > Adrien. In my case I want to ignore document frequency completely. Any > > suggestion to achieve this? > > > > I'm a taker of any solution as this looks like a show stopper for us, so > > even a workaround would help. > > > > I can try to create this other rewrite method you mentioned if you could > > point me in the right direction. > > > > Thanks > > > > On Thursday, January 15, 2015 at 7:44:57 AM UTC-8, Adrien Grand wrote: > > > > > This is because the score takes two factors into account: the document > > > frequency and the edit distance. Quite likely in your case, even though > > > Boss is closer than Bose, Bose has a much lower document frequency which > > > helped it eventually get a better score. I guess we should have another > > > rewrite method that would not take freqs into account (or somehow merge > > > them) to avoid that issue. > > > > > > On Thu, Jan 15, 2015 at 4:06 PM, Eylon Steiner [eylon....@gmail.com](mailto:eylon....@gmail.com) > > > wrote: > > > > > > > Any ideas? > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > Adrien Grand -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/cbfa04e8-afc4-46fe-b945-d006f89ca90f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/cbfa04e8-afc4-46fe-b945-d006f89ca90f%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Mark\_Harwood\_2](https://avatars.discourse-cdn.com/v4/letter/m/c77e96/32.png) [@Mark\_Harwood\_2](https://discuss.elastic.co/u/Mark_Harwood_2)\ **Post date:** [January 20, 2015, 9:11am UTC](https://discuss.elastic.co/t/when-searching-for-boss-with-fuzziness-get-higher-score-for-bose-than-boss-how-comes/21662/6 "2015-01-20T09:11:05Z")
> it doesn't seem like this would address the IDF Trust me, I wrote it. On Tuesday, January 20, 2015 at 12:16:44 AM UTC, [kasper...@yahoo.com](mailto:kasper...@yahoo.com) wrote: > Thanks Mark. Sounds like this issue affects a lot of people. > > I looked at your suggestion about FLT, and the ignore\_tf parameter should > help, however unless I'm missing something, it doesn't seem like this would > address the IDF, and results could be biased. But I will experiment. > > Ultimately I think what my particular use case requires is a scorer that > only uses edit distance (when querying with fuzziness) and field boosts, > but no TF / IDF. > > On Monday, January 19, 2015 at 3:15:47 PM UTC-8, Mark Harwood wrote: > > > This issue rounds up a bunch of related issues that have been raised > > previously: [Wrap stacked tokens in `match` query in a BlendedTerms query for better scoring · Issue #9103 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/9103) > > > > For now try FuzzyLikeThis ( > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-flt-query.html#query-dsl-flt-query) > > ) > > It blends More Like This and fuzzy functionality but includes the > > adjustments to IDF that I think make more sense than the other > > implementations with their bias towards rewarding scarcity. > > > > On Monday, January 19, 2015 at 6:48:49 PM UTC, [kasper...@yahoo.com](mailto:kasper...@yahoo.com) wrote: > > > > > I have the same problem, where some results with higher edit distance > > > are ranked higher than other results that are closer in terms of edit > > > distance. > > > > > > I suspect it does have to do with document frequency, as you think > > > Adrien. In my case I want to ignore document frequency completely. Any > > > suggestion to achieve this? > > > > > > I'm a taker of any solution as this looks like a show stopper for us, so > > > even a workaround would help. > > > > > > I can try to create this other rewrite method you mentioned if you could > > > point me in the right direction. > > > > > > Thanks > > > > > > On Thursday, January 15, 2015 at 7:44:57 AM UTC-8, Adrien Grand wrote: > > > > > > > This is because the score takes two factors into account: the document > > > > frequency and the edit distance. Quite likely in your case, even though > > > > Boss is closer than Bose, Bose has a much lower document frequency which > > > > helped it eventually get a better score. I guess we should have another > > > > rewrite method that would not take freqs into account (or somehow merge > > > > them) to avoid that issue. > > > > > > > > On Thu, Jan 15, 2015 at 4:06 PM, Eylon Steiner [eylon....@gmail.com](mailto:eylon....@gmail.com) > > > > wrote: > > > > > > > > > Any ideas? > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > To view this discussion on the web visit > > > > > [https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com) > > > > > [https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > . > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > -- > > > > Adrien Grand -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/9523b3d5-ffea-4760-9782-69167b9807ed%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/9523b3d5-ffea-4760-9782-69167b9807ed%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Itamar\_Syn\_Hershko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/itamar_syn_hershko/32/725_2.png) [@Itamar\_Syn\_Hershko](https://discuss.elastic.co/u/Itamar_Syn_Hershko)\ **Post date:** [January 20, 2015, 9:11am UTC](https://discuss.elastic.co/t/when-searching-for-boss-with-fuzziness-get-higher-score-for-bose-than-boss-how-comes/21662/7 "2015-01-20T09:11:49Z")
Famous last words 🙂 -- Itamar Syn-Hershko [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) Freelance Developer & Consultant [Lucene.NET](http://Lucene.NET) committer and PMC member On Tue, Jan 20, 2015 at 11:11 AM, Mark Harwood \< [mark.harwood@elasticsearch.com](mailto:mark.harwood@elasticsearch.com)\> wrote: > > it doesn't seem like this would address the IDF > > Trust me, I wrote it. > > On Tuesday, January 20, 2015 at 12:16:44 AM UTC, [kasper...@yahoo.com](mailto:kasper...@yahoo.com) > wrote: > > > Thanks Mark. Sounds like this issue affects a lot of people. > > > > I looked at your suggestion about FLT, and the ignore\_tf parameter should > > help, however unless I'm missing something, it doesn't seem like this would > > address the IDF, and results could be biased. But I will experiment. > > > > Ultimately I think what my particular use case requires is a scorer that > > only uses edit distance (when querying with fuzziness) and field boosts, > > but no TF / IDF. > > > > On Monday, January 19, 2015 at 3:15:47 PM UTC-8, Mark Harwood wrote: > > > > > This issue rounds up a bunch of related issues that have been raised > > > previously: [Wrap stacked tokens in `match` query in a BlendedTerms query for better scoring · Issue #9103 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/9103) > > > > > > For now try FuzzyLikeThis ([http://www.elasticsearch.org/](http://www.elasticsearch.org/) > > > guide/en/elasticsearch/reference/current/query-dsl- > > > flt-query.html#query-dsl-flt-query ) > > > It blends More Like This and fuzzy functionality but includes the > > > adjustments to IDF that I think make more sense than the other > > > implementations with their bias towards rewarding scarcity. > > > > > > On Monday, January 19, 2015 at 6:48:49 PM UTC, [kasper...@yahoo.com](mailto:kasper...@yahoo.com) > > > wrote: > > > > > > > I have the same problem, where some results with higher edit distance > > > > are ranked higher than other results that are closer in terms of edit > > > > distance. > > > > > > > > I suspect it does have to do with document frequency, as you think > > > > Adrien. In my case I want to ignore document frequency completely. Any > > > > suggestion to achieve this? > > > > > > > > I'm a taker of any solution as this looks like a show stopper for us, > > > > so even a workaround would help. > > > > > > > > I can try to create this other rewrite method you mentioned if you > > > > could point me in the right direction. > > > > > > > > Thanks > > > > > > > > On Thursday, January 15, 2015 at 7:44:57 AM UTC-8, Adrien Grand wrote: > > > > > > > > > This is because the score takes two factors into account: the document > > > > > frequency and the edit distance. Quite likely in your case, even though > > > > > Boss is closer than Bose, Bose has a much lower document frequency which > > > > > helped it eventually get a better score. I guess we should have another > > > > > rewrite method that would not take freqs into account (or somehow merge > > > > > them) to avoid that issue. > > > > > > > > > > On Thu, Jan 15, 2015 at 4:06 PM, Eylon Steiner [eylon....@gmail.com](mailto:eylon....@gmail.com) > > > > > wrote: > > > > > > > > > > > Any ideas? > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to the Google > > > > > > Groups "elasticsearch" group. > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > > > msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5% > > > > > > [40googlegroups.com](http://40googlegroups.com) > > > > > > [https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > -- > > > > > Adrien Grand > > > > > > > > -- > > > > You received this message because you are subscribed to the Google Groups > > > > "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send an > > > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/9523b3d5-ffea-4760-9782-69167b9807ed%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/9523b3d5-ffea-4760-9782-69167b9807ed%40googlegroups.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/9523b3d5-ffea-4760-9782-69167b9807ed%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/9523b3d5-ffea-4760-9782-69167b9807ed%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAHTr4Zt5ycYCcwVkqL%2BMazATz5nS5VXtDq6DHmUv2KS%2BrKE\_SQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAHTr4Zt5ycYCcwVkqL%2BMazATz5nS5VXtDq6DHmUv2KS%2BrKE_SQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Mark\_Harwood\_2](https://avatars.discourse-cdn.com/v4/letter/m/c77e96/32.png) [@Mark\_Harwood\_2](https://discuss.elastic.co/u/Mark_Harwood_2)\ **Post date:** [January 20, 2015, 9:20am UTC](https://discuss.elastic.co/t/when-searching-for-boss-with-fuzziness-get-higher-score-for-bose-than-boss-how-comes/21662/8 "2015-01-20T09:20:58Z")
🙂 It's been nearly 10 years but I took a quick look at the code and the IDF balancing stuff is still in there. Testing queries against a large index of cars Lucene's standard fuzzy query on ford~ still has top matches that aren't Fords. FLT works fine. On Tuesday, January 20, 2015 at 9:11:56 AM UTC, Itamar Syn-Hershko wrote: > Famous last words 🙂 > > -- > > Itamar Syn-Hershko > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) > Freelance Developer & Consultant > [Lucene.NET](http://Lucene.NET) committer and PMC member > > On Tue, Jan 20, 2015 at 11:11 AM, Mark Harwood \< > [mark.h...@elasticsearch.com](mailto:mark.h...@elasticsearch.com) \\> wrote: > > > > it doesn't seem like this would address the IDF > > > > Trust me, I wrote it. > > > > On Tuesday, January 20, 2015 at 12:16:44 AM UTC, [kasper...@yahoo.com](mailto:kasper...@yahoo.com) > > wrote: > > > > > Thanks Mark. Sounds like this issue affects a lot of people. > > > > > > I looked at your suggestion about FLT, and the ignore\_tf parameter > > > should help, however unless I'm missing something, it doesn't seem like > > > this would address the IDF, and results could be biased. But I will > > > experiment. > > > > > > Ultimately I think what my particular use case requires is a scorer that > > > only uses edit distance (when querying with fuzziness) and field boosts, > > > but no TF / IDF. > > > > > > On Monday, January 19, 2015 at 3:15:47 PM UTC-8, Mark Harwood wrote: > > > > > > > This issue rounds up a bunch of related issues that have been raised > > > > previously: [Wrap stacked tokens in `match` query in a BlendedTerms query for better scoring · Issue #9103 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/9103) > > > > > > > > For now try FuzzyLikeThis ([http://www.elasticsearch.org/](http://www.elasticsearch.org/) > > > > guide/en/elasticsearch/reference/current/query-dsl- > > > > flt-query.html#query-dsl-flt-query ) > > > > It blends More Like This and fuzzy functionality but includes the > > > > adjustments to IDF that I think make more sense than the other > > > > implementations with their bias towards rewarding scarcity. > > > > > > > > On Monday, January 19, 2015 at 6:48:49 PM UTC, [kasper...@yahoo.com](mailto:kasper...@yahoo.com) > > > > wrote: > > > > > > > > > I have the same problem, where some results with higher edit distance > > > > > are ranked higher than other results that are closer in terms of edit > > > > > distance. > > > > > > > > > > I suspect it does have to do with document frequency, as you think > > > > > Adrien. In my case I want to ignore document frequency completely. Any > > > > > suggestion to achieve this? > > > > > > > > > > I'm a taker of any solution as this looks like a show stopper for us, > > > > > so even a workaround would help. > > > > > > > > > > I can try to create this other rewrite method you mentioned if you > > > > > could point me in the right direction. > > > > > > > > > > Thanks > > > > > > > > > > On Thursday, January 15, 2015 at 7:44:57 AM UTC-8, Adrien Grand wrote: > > > > > > > > > > > This is because the score takes two factors into account: the > > > > > > document frequency and the edit distance. Quite likely in your case, even > > > > > > though Boss is closer than Bose, Bose has a much lower document frequency > > > > > > which helped it eventually get a better score. I guess we should have > > > > > > another rewrite method that would not take freqs into account (or somehow > > > > > > merge them) to avoid that issue. > > > > > > > > > > > > On Thu, Jan 15, 2015 at 4:06 PM, Eylon Steiner [eylon....@gmail.com](mailto:eylon....@gmail.com) > > > > > > wrote: > > > > > > > > > > > > > Any ideas? > > > > > > > > > > > > > > -- > > > > > > > You received this message because you are subscribed to the Google > > > > > > > Groups "elasticsearch" group. > > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > > > To view this discussion on the web visit > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/52e09e54-](https://groups.google.com/d/msgid/elasticsearch/52e09e54-) > > > > > > > 90b6-4014-8454-34e3db5756e5%[40googlegroups.com](http://40googlegroups.com) > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/52e09e54-90b6-4014-8454-34e3db5756e5%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > > > . > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > > > -- > > > > > > Adrien Grand > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google Groups > > > > > "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send an > > > > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > > > To view this discussion on the web visit > > > > > [https://groups.google.com/d/msgid/elasticsearch/9523b3d5-ffea-4760-9782-69167b9807ed%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/9523b3d5-ffea-4760-9782-69167b9807ed%40googlegroups.com) > > > > > [https://groups.google.com/d/msgid/elasticsearch/9523b3d5-ffea-4760-9782-69167b9807ed%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/9523b3d5-ffea-4760-9782-69167b9807ed%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > . > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/84f178ba-d150-4b7d-9a54-b419bc962499%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/84f178ba-d150-4b7d-9a54-b419bc962499%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:37am UTC](https://discuss.elastic.co/t/when-searching-for-boss-with-fuzziness-get-higher-score-for-bose-than-boss-how-comes/21662/9 "2017-07-06T00:37:49Z")