# Fuzz search thrown too complex to determinize exception

**URL:** https://discuss.elastic.co/t/fuzz-search-thrown-too-complex-to-determinize-exception/117547
**Category:** Elasticsearch
**Created:** [January 29, 2018, 10:52pm UTC](https://discuss.elastic.co/t/fuzz-search-thrown-too-complex-to-determinize-exception/117547 "2018-01-29T22:52:51Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![Youxu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/youxu/32/117375_2.png) [@Youxu](https://discuss.elastic.co/u/Youxu)
#### Post date: [January 29, 2018, 10:52pm UTC](https://discuss.elastic.co/t/fuzz-search-thrown-too-complex-to-determinize-exception/117547/1 "2018-01-29T22:52:51Z")

</div>

We saw some too\_complex\_to\_determinize\_exception when doing fuzz search.

Exception details:  
`too_complex_to_determinize_exception: Determinizing automaton with 41207 states and 74541 transitions would result in more than 10000 states., ES Status: 500\`

Following is the example query DSL which caused the exception:

```auto
"query": {
  "multi_match": {
    "query": "nous savons que vous voulez vraiment commencer dès maintenant, mais vous allez devoir patienter un peu. recherchez dans le windows store la date de lancement.",
    "fuzziness": "AUTO",
    "fields": ["Term"]
  }
}

```

Basically, the exception occurred when the query is a long length query.  
But if I removed the "fuzziness" from the query DSL, then no exception.

And following is the analysis settings:

```auto
{
 "analysis": {
   "filter": {
     "whitespace_normalization": {
       "pattern":"\\s+",
       "type":"pattern_replace",
       "replacement":""
      }
    },
   "analyzer": {
     "keyword_ngram_suggest": {
       "filter": [
         "lowercase",
         "whitespace_normalization",
         "ngram_filter"
        ],
       "type":"custom",
       "tokenizer":"keyword"
      },
     "lowercase_norm_keyword": {
       "filter": [
         "lowercase",
         "whitespace_normalization",
         "trim"
        ],
       "type":"custom",
       "tokenizer":"keyword"
      }
    }
  }
}

```

And field mappings:

```auto
"Term": {
  "type": "text",
  "analyzer": "keyword_ngram_suggest",
  "search_analyzer": "lowercase_norm_keyword"
}

```

I guess the reason is because in my setting, the whole input query is treated as a single token, and fuzz search cannot handle long length token well.

My question is, is there any soft limit setting which I can increase the threshold so that allow longer token when dong fuzz search?

---

<div class="post-metadata">

### Author: ![AlanWoodward](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alanwoodward/32/41962_2.png) [@AlanWoodward](https://discuss.elastic.co/u/AlanWoodward)
#### Post date: [January 30, 2018, 1:14pm UTC](https://discuss.elastic.co/t/fuzz-search-thrown-too-complex-to-determinize-exception/117547/2 "2018-01-30T13:14:25Z")

</div>

Hi @Youxu,

There isn't a setting in elasticsearch to lift the cap on determinized states, no (in fact, there isn't even a way of doing it in Lucene at the moment via FuzzyQuery). I wouldn't recommend doing it anyway, as it would be a very inefficient way of searching. Is there a reason why you aren't breaking things up on whitespace and doing something like a minimum-should-match query here?

---

<div class="post-metadata">

### Author: ![Youxu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/youxu/32/117375_2.png) [@Youxu](https://discuss.elastic.co/u/Youxu)
#### Post date: [February 8, 2018, 5:57pm UTC](https://discuss.elastic.co/t/fuzz-search-thrown-too-complex-to-determinize-exception/117547/3 "2018-02-08T17:57:04Z")

</div>

I am implementing a auto complete feature using Elasticsearch. Our requirement is, for any user input text in search box, only those terms starts with the user input text (allow typo and redundant whitespaces) appears in suggested terms list.

Some examples:

Suppose there are 2 terms in index:

google account  
sign in with google account  
how to sign in google account

when user input "g", "go" or "goo", only "google account" appears in terms list.  
when user input "si", "sign in", or "sig in", only sign in with google account appears.  
when user input "sign with", nothing appears.

That is why I defined the Term field as key word with ngram filter and whitespace\_normalization filter.

Do you have alternative solutions for our starts with auto complete?

---

<div class="post-metadata">

### Author: ![AlanWoodward](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alanwoodward/32/41962_2.png) [@AlanWoodward](https://discuss.elastic.co/u/AlanWoodward)
#### Post date: [February 15, 2018, 2:29pm UTC](https://discuss.elastic.co/t/fuzz-search-thrown-too-complex-to-determinize-exception/117547/4 "2018-02-15T14:29:42Z")

</div>

Hi @Youxu

Could you use a 'truncate' filter to limit token length? Putting in an entire sentence of text is presumably an edge case, most of the time you're expecting people to type a few characters, correct? So truncating things to say 100 characters should leave the vast majority of users unaffected, while preventing errors for those who copy and paste large amounts of text in.

---

<div class="post-metadata">

### Author: ![Youxu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/youxu/32/117375_2.png) [@Youxu](https://discuss.elastic.co/u/Youxu)
#### Post date: [February 17, 2018, 10:03pm UTC](https://discuss.elastic.co/t/fuzz-search-thrown-too-complex-to-determinize-exception/117547/5 "2018-02-17T22:03:26Z")

</div>

thanks your suggest, I think it makes sense!

发自我的 iPhone

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 17, 2018, 10:03pm UTC](https://discuss.elastic.co/t/fuzz-search-thrown-too-complex-to-determinize-exception/117547/6 "2018-03-17T22:03:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
