# Fuzzy Search not working as intended **URL:** \ **Category:** Elasticsearch\ **Created:** [March 29, 2017, 1:38pm UTC](https://discuss.elastic.co/t/fuzzy-search-not-working-as-intended/80489 "2017-03-29T13:38:32Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![amlanrath](https://avatars.discourse-cdn.com/v4/letter/a/d6d6ee/32.png) [@amlanrath](https://discuss.elastic.co/u/amlanrath)\ **Post date:** [March 29, 2017, 1:38pm UTC](https://discuss.elastic.co/t/fuzzy-search-not-working-as-intended/80489/1 "2017-03-29T13:38:32Z")
I changed the default Fuzziness logic of ES from Fuzziness.AUTO to my own which is as follows - 0-3 - no edits allowed or Fuzziness.ZERO 4-10 - 1 edit allowed or Fuzziness.ONE 10+ - 2 edits allowed or Fuzziness.TWO Earlier the way fuzzy was running was - MultiMatchQueryBuilder queryBuilder2 = QueryBuilders.multiMatchQuery(QueryParser.escape(suggestion), FULL\_SEARCH\_FIELDS).fuzziness(Fuzziness.AUTO).lenient(true).boost(0.1f); After this change I also added different boosts to each field required as follows - QueryStringQueryBuilder queryBuilder2 = QueryBuilders.queryStringQuery(QueryParser.escape(suggestion)).analyzer("hsproduct").fuzziness(fuzziness).lenient(true); for (ProductFuzzySearchKeywordEnum field : ProductFuzzySearchKeywordEnum.values()) { queryBuilder2.field(field.getFieldName(), field.getFieldBoost()); } After the above changes, the FUZZY isn't matching some basic things like "SCHOOL" with search "SCOOL" and "FOOTWEAR" with "FOOTWEAT". I am currently on ES 5.1 and would appreciate if anyone can find something wrong with what I have done. ---
**Author:** ![dakrone](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dakrone/32/23351_2.png) [@dakrone](https://discuss.elastic.co/u/dakrone)\ **Post date:** [March 29, 2017, 4:10pm UTC](https://discuss.elastic.co/t/fuzzy-search-not-working-as-intended/80489/2 "2017-03-29T16:10:51Z")
What is your `hsproduct` analyzer doing? Can you paste the configuration for it? ---
**Author:** ![venkata\_sreekanth\_bh](https://avatars.discourse-cdn.com/v4/letter/v/65b543/32.png) [@venkata\_sreekanth\_bh](https://discuss.elastic.co/u/venkata_sreekanth_bh)\ **Post date:** [March 29, 2017, 5:06pm UTC](https://discuss.elastic.co/t/fuzzy-search-not-working-as-intended/80489/3 "2017-03-29T17:06:49Z")
try indexing the field with nGram analyzer. It generally yields better results then Fuzzy. set the mingram to smallest length of your string and max gram to largest length of your string. ---
**Author:** ![amlanrath](https://avatars.discourse-cdn.com/v4/letter/a/d6d6ee/32.png) [@amlanrath](https://discuss.elastic.co/u/amlanrath)\ **Post date:** [March 30, 2017, 12:34am UTC](https://discuss.elastic.co/t/fuzzy-search-not-working-as-intended/80489/4 "2017-03-30T00:34:18Z")
The hsproduct analyzer is as follows - {"settings": { "analysis": { "analyzer": { "hsproduct": { "type": "custom", "filter": ["standard", "lowercase", "stopword", "synonym", "stemmer"], "tokenizer": "standard", "char\_filter": ["input\_char\_filter"] } }, "filter": { "stopword": { "type": "stop", "ignore\_case": "true" }, "stemmer": { "type": "stemmer", "language": "english" }, "synonym": { "type": "synonym", "synonyms":["hi,Hi"] } }, "char\_filter": { "input\_char\_filter": { "type": "pattern\_replace", "pattern": "[^A-z0-9]+", "replacement": "" } } } } } ---
**Author:** ![amlanrath](https://avatars.discourse-cdn.com/v4/letter/a/d6d6ee/32.png) [@amlanrath](https://discuss.elastic.co/u/amlanrath)\ **Post date:** [March 30, 2017, 12:39am UTC](https://discuss.elastic.co/t/fuzzy-search-not-working-as-intended/80489/5 "2017-03-30T00:39:44Z")
But doesn't Fuzzy Search match any and all strings corresponding to a particular Damaeu-Levenshtein distance, like matching SCOOL with SCHOOL if edit distance is defined as 1 and not matching COOL with SCHOOL as it required an edit distance of 2 ? Or is there any limitation of Fuzzy Search in ES that am not aware of ? Please let me know of any such limitation if there exists one. Also Ngram tokenizers as per my knowledge are more useful for languages which very long compound words as per elasticsearch official site - "The ngram tokenizer first breaks text down into words whenever it encounters one of a list of specified characters, then it emits N-grams of each word of the specified length. N-grams are like a sliding window that moves across the word - a continuous sequence of characters of the specified length. They are useful for querying languages that don’t use spaces or that have long compound words, like German.", taken from [https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-ngram-tokenizer.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-ngram-tokenizer.html) Also in my case the min\_gram would range from 1-2 and max\_gram would go till 10-15. This would produce a lot of tokens for each indexed field. Your views regarding this ? ---
**Author:** ![venkata\_sreekanth\_bh](https://avatars.discourse-cdn.com/v4/letter/v/65b543/32.png) [@venkata\_sreekanth\_bh](https://discuss.elastic.co/u/venkata_sreekanth_bh)\ **Post date:** [March 30, 2017, 12:55am UTC](https://discuss.elastic.co/t/fuzzy-search-not-working-as-intended/80489/6 "2017-03-30T00:55:54Z")
You are assuming that fuzzy would replace the characters you are expecting but the offset could be anywhere. ---
**Author:** ![amlanrath](https://avatars.discourse-cdn.com/v4/letter/a/d6d6ee/32.png) [@amlanrath](https://discuss.elastic.co/u/amlanrath)\ **Post date:** [March 30, 2017, 1:00am UTC](https://discuss.elastic.co/t/fuzzy-search-not-working-as-intended/80489/7 "2017-03-30T01:00:00Z")
Ok, can you let me know what do you mean by this "offset" you are talking about and how it affects fuzzy searches ? I didn't know there existed such a thing. Also I have edited my earlier reply, can you have a look at that. ---
**Author:** ![venkata\_sreekanth\_bh](https://avatars.discourse-cdn.com/v4/letter/v/65b543/32.png) [@venkata\_sreekanth\_bh](https://discuss.elastic.co/u/venkata_sreekanth_bh)\ **Post date:** [March 30, 2017, 1:33am UTC](https://discuss.elastic.co/t/fuzzy-search-not-working-as-intended/80489/8 "2017-03-30T01:33:02Z")
offset meaning the positions of characters. Fuzzy allows for transposition, deletion, insertion and substitution for example 'abc' with fuzziness one will return 'acb' as also a valid result. In you example "SCHOOL" with search "SCOOL" can match SDOOL, SCOOL SCHOOD etc. -- nGrams may not help here, regex queries will help more. [https://www.elastic.co/guide/en/elasticsearch/reference/5.2/query-dsl-regexp-query.html](https://www.elastic.co/guide/en/elasticsearch/reference/5.2/query-dsl-regexp-query.html) "FOOTWEAR" with "FOOTWEAT" nGrams will match here. I also noticed you are setting a boost of 0.1f which is punishing for matching. Did you really want to boost use a value greater than 1. less than one lower the score. nGrams will produce lot of space and take quite a bit of diskspace too. But your queries will run faster than wildcard or fuzzy. ---
**Author:** ![amlanrath](https://avatars.discourse-cdn.com/v4/letter/a/d6d6ee/32.png) [@amlanrath](https://discuss.elastic.co/u/amlanrath)\ **Post date:** [March 30, 2017, 5:27am UTC](https://discuss.elastic.co/t/fuzzy-search-not-working-as-intended/80489/9 "2017-03-30T05:27:19Z")
Ok, I got why ngrams would be faster, but I am sorry I still didn't get why SCOOL will not match SCHOOL as insertion of a H should be allowed in fuzzy. Also since my highest relevance matching fields have a boost of 0.9f, I have adjust the boosting of other fields accordingly so that they don't get more relevancy than that one. Also why is it punishing to use a boost of 0.1f if all my other fields also use boosts in similar range as that ? ---
**Author:** ![venkata\_sreekanth\_bh](https://avatars.discourse-cdn.com/v4/letter/v/65b543/32.png) [@venkata\_sreekanth\_bh](https://discuss.elastic.co/u/venkata_sreekanth_bh)\ **Post date:** [March 30, 2017, 1:45pm UTC](https://discuss.elastic.co/t/fuzzy-search-not-working-as-intended/80489/10 "2017-03-30T13:45:23Z")
If you want boost a certain result use boost greater than one, if you want lower the relevancy use boost between 0-1. Please read documentation regarding this. ---
**Author:** ![dakrone](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dakrone/32/23351_2.png) [@dakrone](https://discuss.elastic.co/u/dakrone)\ **Post date:** [March 30, 2017, 6:56pm UTC](https://discuss.elastic.co/t/fuzzy-search-not-working-as-intended/80489/11 "2017-03-30T18:56:41Z")
I tried reproducing your issue with HTTP requests, but I could not reproduce it. The following correctly matches the "school" document when searching for "scool": ``` DELETE i PUT /i { "settings": { "analysis": { "analyzer": { "hsproduct": { "type": "custom", "filter": ["standard", "lowercase", "stopword", "synonym", "stemmer"], "tokenizer": "standard", "char_filter": ["input_char_filter"] } }, "filter": { "stopword": { "type": "stop", "ignore_case": "true" }, "stemmer": { "type": "stemmer", "language": "english" }, "synonym": { "type": "synonym", "synonyms":["hi,Hi"] } }, "char_filter": { "input_char_filter": { "type": "pattern_replace", "pattern": "[^A-z0-9]+", "replacement": "" } } } }, "mappings": { "doc": { "properties": { "foo": { "type": "text", "analyzer": "hsproduct" } } } } } POST /i/_analyze { "field": "foo", "text": "school" } POST /i/doc/1 {"foo": "school"} // Matches the document POST /i/_search { "query": { "multi_match": { "query": "scool", "fields": ["foo"], "fuzziness": 1 } } } ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 27, 2017, 6:56pm UTC](https://discuss.elastic.co/t/fuzzy-search-not-working-as-intended/80489/12 "2017-04-27T18:56:44Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.