"match_phrase_prefix" TooManyClauses[maxClauseCount is set to 1024]

Elastic Stack Elasticsearch
1

when i use "match_phrase_prefix" query in my project.

the query is :

{
  "query": {
    "match_phrase_prefix": {
      "nickName": "a"
    }
  }
}

but it throws the exception as follow:

org.elasticsearch.action.search.SearchPhaseExecutionException: Failed to execute phase [dfs], all shards failed; shardFailures {[J4HFhidITlKtvOlSf2B36A][user][0]: DfsPhaseExecutionException[[user][0]: query[filtered(nickName:"a*"~1)->cache(_type:user)],from[0],size[10]: Dfs Failed [Exception during dfs phase]]; nested: TooManyClauses[maxClauseCount is set to 1024]; }{[J4HFhidITlKtvOlSf2B36A][user][1]: DfsPhaseExecutionException[[user][1]: query[filtered(nickName:"a*"~1)->cache(_type:user)],from[0],size[10]: Dfs Failed [Exception during dfs phase]]; nested: TooManyClauses[maxClauseCount is set to 1024]; }{[porfJBUXRWeSygtrF_rjRg][user][2]: RemoteTransportException[[mq-03][inet[/10.163.6.246:9300]][indices:data/read/search[phase/dfs]]]; nested: DfsPhaseExecutionException[[user][2]: query[filtered(nickName:"a*"~1)->cache(_type:user)],from[0],size[10]: Dfs Failed [Exception during dfs phase]]; nested: TooManyClauses[maxClauseCount is set to 1024]; }{[-kFmY4FhSDe0s1g-j0cn4w][user][3]: RemoteTransportException[[mq-01][inet[/10.162.207.86:9300]][indices:data/read/search[phase/dfs]]]; nested: DfsPhaseExecutionException[[user][3]: query[filtered(nickName:"a*"~1)->cache(_type:user)],from[0],size[10]: Dfs Failed [Exception during dfs phase]]; nested: TooManyClauses[maxClauseCount is set to 1024]; }{[J4HFhidITlKtvOlSf2B36A][user][4]: DfsPhaseExecutionException[[user][4]: query[filtered(nickName:"a*"~1)->cache(_type:user)],from[0],size[10]: Dfs Failed [Exception during dfs phase]]; nested: TooManyClauses[maxClauseCount is set to 1024]; }{[klXNljLDRMKty9yIdOZp6A][user][5]: RemoteTransportException[[design-mq-04][inet[/10.173.34.187:9300]][indices:data/read/search[phase/dfs]]]; nested: DfsPhaseExecutionException[[user][5]: query[filtered(nickName:"a*"~1)->cache(_type:user)],from[0],size[10]: Dfs Failed [Exception during dfs phase]]; nested: TooManyClauses[maxClauseCount is set to 1024]; }{[fu_kgQlKQq2iemZ3v-TKkg][user][6]: RemoteTransportException[[design-mq-03][inet[/10.172.135.214:9300]][indices:data/read/search[phase/dfs]]]; nested: DfsPhaseExecutionException[[user][6]: query[filtered(nickName:"a*"~1)->cache(_type:user)],from[0],size[10]: Dfs Failed [Exception during dfs phase]]; nested: TooManyClauses[maxClauseCount is set to 1024]; }{[T3C8QuLGSZushd71dOHGeg][user][7]: RemoteTransportException[[mq-04][inet[/10.172.187.174:9300]][indices:data/read/search[phase/dfs]]]; nested: DfsPhaseExecutionException[[user][7]: query[filtered(nickName:"a*"~1)->cache(_type:user)],from[0],size[10]: Dfs Failed [Exception during dfs phase]]; nested: TooManyClauses[maxClauseCount is set to 1024]; }{[T3C8QuLGSZushd71dOHGeg][user][8]: RemoteTransportException[[mq-04][inet[/10.172.187.174:9300]][indices:data/read/search[phase/dfs]]]; nested: DfsPhaseExecutionException[[user][8]: query[filtered(nickName:"a*"~1)->cache(_type:user)],from[0],size[10]: Dfs Failed [Exception during dfs phase]]; nested: TooManyClauses[maxClauseCount is set to 1024]; }{[klXNljLDRMKty9yIdOZp6A][user][9]: RemoteTransportException[[design-mq-04][inet[/10.173.34.187:9300]][indices:data/read/search[phase/dfs]]]; nested: DfsPhaseExecutionException[[user][9]: query[filtered(nickName:"a*"~1)->cache(_type:user)],from[0],size[10]: Dfs Failed [Exception during dfs phase]]; nested: TooManyClauses[maxClauseCount is set to 1024]; }{[porfJBUXRWeSygtrF_rjRg][user][10]: RemoteTransportException[[mq-03][inet[/10.163.6.246:9300]][indices:data/read/search[phase/dfs]]]; nested: DfsPhaseExecutionException[[user][10]: query[filtered(nickName:"a*"~1)->cache(_type:user)],from[0],size[10]: Dfs Failed [Exception during dfs phase]]; nested: TooManyClauses[maxClauseCount is set to 1024]; }{[klXNljLDRMKty9yIdOZp6A][user][11]: RemoteTransportException[[design-mq-04][inet[/10.173.34.187:9300]][indices:data/read/search[phase/dfs]]]; nested: DfsPhaseExecutionException[[user][11]: query[filtered(nickName:"a*"~1)->cache(_type:user)],from[0],size[10]: Dfs Failed [Exception during dfs phase]]; nested: TooManyClauses[maxClauseCount is set to 1024]; }

how can i solve it ?

2

Hi @felayman,

it looks the query is rewritten internally into a large number of boolean queries. While you could increase the limit of 1024 boolean clauses I think this is the wrong route here.

I think there are a number of things to consider here:

  1. You're searching for user names. You should not analyze them but rather store them as is (i.e. map them as keyword instead of text (ES 5.0 and later)). So I think you should be using a prefix query to begin with.
  2. Both the prefix query and the phrase prefix query are very resource intensive. If you want to use this for auto-complete or suggestion there are alternatives (see also the partial matching chapter in the Definitive Guide for more context).

If I'm wrong and you really need to use a phrase prefix query, you should limit the number of expansions with the max_expansions parameter (see docs) and/or expect the user to enter at least 2 or 3 characters and only then start to hit Elasticsearch.

Daniel

3

@danielmitterdorfer

First at all , so thanks for your reply.

i try to use the max_expansions(see doc) parameter to control to how many suffixes the last term will be expanded. but it doesn't work as expected.

and now i use the prefix query instead of match_phrase_prefix query .

4

To which value(s) did you set max_expansions (IIRC the default value is 50)? Can you explain in which way it does not work as expected?

Does this mean that:

  1. You have changed your mapping of the field nickName
  2. You changed your use of match_phrase_prefix query to prefix query
  3. It is working now?

Daniel

5

1.i don't change my mapping.
2.yes,i change my use of match_phrase_prefix query to prefix query
3.now it works well.

Thanks all ,Daniel.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.