Context Suggester is very slow for large context set

[raychow]

Hi,

I'm considering to use context suggester to auto complete user input, and the context is used as permission scope:

PUT localhost:9200/multi_conditions_2?pretty
Content-Type: application/json

{
  "mappings": {
    "properties": {
      "permission_scope": {
        "type": "keyword"
      },
      "id_search": {
        "type": "text",
        "fields": {
          "suggest": {
            "type": "completion",
            "contexts": [
              {
                "name": "permission_scope",
                "type": "category",
                "path": "permission_scope"
              }
            ]
          }
        }
      }
    }
  }
}

Since one user may have many permissions, I found that the speed of context suggester slowed down significantly as the number of search contexts increased:

{
  "_source": "id_search",
  "suggest": {
    "id_search": {
      "text": "sg-",
      "completion": {
        "field": "id_search.suggest",
        "skip_duplicates": true,
        "contexts": {
          "permission_scope": [
            "123456789.permission1.sub-permission1.leaf-permission1",
            "123456789.permission2.sub-permission1.leaf-permission1",
            "123456789.permission3.sub-permission1.leaf-permission1",
            "123456789.permission4.sub-permission1.leaf-permission1",
            "123456789.permission5.sub-permission1.leaf-permission1",
            ... (200 contexts in total)
            "123456789.permission200.sub-permission1.leaf-permission1"
          ]
        }
      }
    }
  }
}

When the number of search contexts exceeds a certain threshold, the search fails and returns an error such as:

{
  "type": "too_complex_to_determinize_exception",
  "reason": "Determinizing automaton with 42901 states and 44900 transitions would result in more than 10000 states."
}

Is there any solution to avoid this error and speed up the query?