# Fuzzy search "too complex to determinize exception" with unicode characters

**URL:** https://discuss.elastic.co/t/fuzzy-search-too-complex-to-determinize-exception-with-unicode-characters/168882
**Category:** Elasticsearch
**Created:** [February 18, 2019, 5:21pm UTC](https://discuss.elastic.co/t/fuzzy-search-too-complex-to-determinize-exception-with-unicode-characters/168882 "2019-02-18T17:21:20Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![CathalCoffey](https://avatars.discourse-cdn.com/v4/letter/c/3e96dc/32.png) [@CathalCoffey](https://discuss.elastic.co/u/CathalCoffey)
#### Post date: [February 18, 2019, 5:21pm UTC](https://discuss.elastic.co/t/fuzzy-search-too-complex-to-determinize-exception-with-unicode-characters/168882/1 "2019-02-18T17:21:20Z")

</div>

We occasionally see the following exception. It seems to be dependant on whether the search contains unicode characters or not.

```auto
{
  "error": {
    "root_cause": [
      {
        "type": "too_complex_to_determinize_exception",
        "reason": "too_complex_to_determinize_exception: Determinizing automaton with 18934 states and 27853 transitions would result in more than 10000 states."
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": "redacted",
        "node": "pxuPrKQMSdq1c1p8Cwrb9Q",
        "reason": {
          "type": "too_complex_to_determinize_exception",
          "reason": "too_complex_to_determinize_exception: Determinizing automaton with 18934 states and 27853 transitions would result in more than 10000 states."
        }
      }
    ]
  },
  "status": 500
}

```

We limit our search input field to 250 characters. If I search for 250 `a` it works fine. However, if I use 250 `วั` we see this exception. Why is this the case? How can I impose a safe limit on my input field if it is dependent on the character encoding?

---

<div class="post-metadata">

### Author: ![CathalCoffey](https://avatars.discourse-cdn.com/v4/letter/c/3e96dc/32.png) [@CathalCoffey](https://discuss.elastic.co/u/CathalCoffey)
#### Post date: [February 19, 2019, 10:48am UTC](https://discuss.elastic.co/t/fuzzy-search-too-complex-to-determinize-exception-with-unicode-characters/168882/2 "2019-02-19T10:48:11Z")

</div>

Here is a more specific example, this time it fails with even fewer characters!

```auto
{
  "query": {
    "bool": {
      "should": [
        {
          "multi_match": {
            "fields": [
              "body_text.analyzed_unigram"
            ],
            "query": "วัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวัวั",
            "operator": "or",
            "fuzziness": "auto",
            "prefix_length": 3
          }
        }
      ]
    }
  }
}

```

```auto
{
  "error": {
    "root_cause": [
      {
        "type": "too_complex_to_determinize_exception",
        "reason": "too_complex_to_determinize_exception: Determinizing automaton with 18934 states and 27853 transitions would result in more than 10000 states."
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": "article_search_production_v2",
        "node": "pxuPrKQMSdq1c1p8Cwrb9Q",
        "reason": {
          "type": "too_complex_to_determinize_exception",
          "reason": "too_complex_to_determinize_exception: Determinizing automaton with 18934 states and 27853 transitions would result in more than 10000 states."
        }
      }
    ]
  },
  "status": 500
}

```

```auto
{
  "query": {
    "bool": {
      "should": [
        {
          "multi_match": {
            "fields": [
              "body_text.analyzed_unigram"
            ],
            "query": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
            "operator": "or",
            "fuzziness": "auto",
            "prefix_length": 3
          }
        }
      ]
    }
  }
}

```

```auto
{"took":6,"timed_out":false,"_shards":{"total":2,"successful":2,"skipped":0,"failed":0},"hits":{"total":202,"max_score":0.0,"hits":[]...

```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 19, 2019, 10:48am UTC](https://discuss.elastic.co/t/fuzzy-search-too-complex-to-determinize-exception-with-unicode-characters/168882/3 "2019-03-19T10:48:12Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
