# Null\_pointer\_exception when use bool query **URL:** \ **Category:** Elasticsearch\ **Created:** [June 21, 2019, 8:33am UTC](https://discuss.elastic.co/t/null-pointer-exception-when-use-bool-query/186831 "2019-06-21T08:33:22Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![davin4u](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davin4u/32/48498_2.png) [@davin4u](https://discuss.elastic.co/u/davin4u)\ **Post date:** [June 21, 2019, 8:33am UTC](https://discuss.elastic.co/t/null-pointer-exception-when-use-bool-query/186831/1 "2019-06-21T08:33:22Z")
Hi everyone. I have the weird issue when i use simple bool query ``` curl -X GET "localhost:9200/people/_search" -H 'Content-Type: application/json' -d' { "query": { "bool": { "should": [ {"match":{"name":"andrew"}} ], "must": [ {"match":{"birth_year":"2005"}} ] } }, "sort":{"_score":{"order":"desc"}},"from":0,"size":"15","min_score":5 } ' | python -m json.tool ``` In this case i get null\_pointer\_exception error. The weird thing is, with different options i may not get it, so, for example with birth\_year = 2019 i don't get it. I thought that this can be because of wrong usage of bool query or wrong settings or something like that, but wasn't managed to find why exactly that happens. I fixed that issue using "filter" clause instead of "must" - this way that works and i don't get the error, but, that is not a solution. ES version which i use: 7.1.0 docker-compose.yml: ``` elasticsearch: image: elasticsearch:7.1.0 restart: always ports: - "9200:9200" - "9300:9300" volumes: - ./esdata71:/usr/share/elasticsearch/data environment: - bootstrap.memory_lock=true - "ES_JAVA_OPTS=-Xms512m -Xmx512m" - discovery.type=single-node ulimits: memlock: soft: -1 hard: -1 ``` Index settings: ``` [ "max_ngram_diff" => 10, "analysis" => [ "tokenizer" => [ "edge_trigrams_tokenizer" => [ "type" => "edge_ngram", "min_gram"=> 3, "max_gram"=> 20, "token_chars" => ["letter", "whitespace", "punctuation", "symbol", "digit"] ], "trigrams_tokenizer" => [ "type" => "ngram", "min_gram"=> 3, "max_gram"=> 10, "token_chars" => ["letter", "whitespace", "punctuation", "symbol", "digit"] ], "short_trigrams_tokenizer" => [ "type" => "edge_ngram", "min_gram" => 1, "max_gram" => 3, "token_chars" => ["letter", "whitespace", "punctuation", "symbol", "digit"] ] ], "analyzer"=> [ "edge_trigrams"=> [ "type"=> "custom", "tokenizer"=> "edge_trigrams_tokenizer", "filter"=> [ "lowercase", "asciifolding" ], "char_filter" => ["synonym"] ], "trigrams"=> [ "type"=> "custom", "tokenizer"=> "trigrams_tokenizer", "filter"=> [ "lowercase", "asciifolding" ], "char_filter" => ["synonym"] ] ], "char_filter" => [ "synonym" => [ "type" => "mapping", "mappings" => [ "v.d => van de", "v/d => van de", "vd => van de", "vh => van het", "v/h => van het", "v.h => van het", "dl => de la", "d/l => de la", "d'la => de la", "du => de le", "de l' => de la" ] ] ] ] ] ``` Mapping settings: ``` "name" => [ 'type' => 'text', 'analyzer' => 'edge_trigrams' ], "last_name" => [ 'type' => 'text', 'analyzer' => 'edge_trigrams' ], "exact_name" => [ 'type' => 'text', 'analyzer' => 'trigrams' ], "birth_year" => [ 'type' => 'text', 'analyzer' => 'standard' ], "sex" => [ 'type' => 'text', 'analyzer' => 'standard' ], "active" => [ 'type' => 'boolean' ] ``` The error which i get: ``` { "error": { "caused_by": { "caused_by": { "reason": null, "type": "null_pointer_exception" }, "reason": null, "type": "null_pointer_exception" }, "failed_shards": [ { "index": "people", "node": "3beKgQxDRxe5V5Ln6ZwmUA", "reason": { "reason": null, "type": "null_pointer_exception" }, "shard": 0 } ], "grouped": true, "phase": "query", "reason": "all shards failed", "root_cause": [ { "reason": null, "type": "null_pointer_exception" } ], "type": "search_phase_execution_exception" }, "status": 500 } ``` Another weird thing, even with the same options (like the same name and birth\_year) but with different "min\_score" in query, in one case i get error but with different "min\_score" i don't get error. I assume that this is something related to score calculation, or to some kind of data manipulation inside the ES engine, so probably something wrong inside my index and that cause an exception in different specific cases, so basically some documents inside my index have something which should not be there. ---
**Author:** ![davin4u](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davin4u/32/48498_2.png) [@davin4u](https://discuss.elastic.co/u/davin4u)\ **Post date:** [June 21, 2019, 2:40pm UTC](https://discuss.elastic.co/t/null-pointer-exception-when-use-bool-query/186831/2 "2019-06-21T14:40:48Z")
**UPDATE:** After investigating i figured out next: That happens when i use "min\_score" in my query Data from Elasticsearch log: ``` "Caused by: java.lang.NullPointerException", "at org.apache.lucene.search.ReqOptSumScorer.setMinCompetitiveScore(ReqOptSumScorer.java:299) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]", "at org.apache.lucene.search.ScoreCachingWrappingScorer.setMinCompetitiveScore(ScoreCachingWrappingScorer.java:59) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]", "at org.apache.lucene.search.TopScoreDocCollector.updateMinCompetitiveScore(TopScoreDocCollector.java:242) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]", "at org.apache.lucene.search.TopScoreDocCollector$SimpleTopScoreDocCollector$1.collect(TopScoreDocCollector.java:78) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]", "at org.elasticsearch.common.lucene.MinimumScoreCollector.collect(MinimumScoreCollector.java:57) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:263) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]", "at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:214) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]", "at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]", "at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:652) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]", "at org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:177) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:443) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]", "at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:275) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:115) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:349) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:393) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.search.SearchService.access$100(SearchService.java:124) ~[elasticsearch-7.1.0.jar:7.1.0]", ``` So, from the top line i noticed that error appears in the file ReqOptSumScorer.java:299 Here is repository: > and on the 299 line i see next: `maxScorePropagator.setMinCompetitiveScore(minScore);` variable is defined on line 54: ``` if (scoreMode == ScoreMode.TOP_SCORES) { this.maxScorePropagator = new MaxScoreSumPropagator(Arrays.asList(reqScorer, optScorer)); } else { this.maxScorePropagator = null; } ``` so i assume that in my case it goes to "else" section and variable is NULL. Then i found info about "score\_mode" in elasticsearch documentation: [https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html) So, there are several possible variants of "score\_mode", but when i change query to this: ``` curl -X GET "localhost:9200/people/_search" -H 'Content-Type: application/json' -d' { "query": { "bool": { "should": [ {"match":{"name":"andrew"}} ], "must": [ {"match":{"birth_year":"2005"}} ] } }, "sort":{"_score":{"order":"desc"}},"from":0,"size":"15","min_score":2,"score_mode":"max" } ' | python -m json.tool ``` i get another error: ``` { "error": { "col": 86, "line": 14, "reason": "Unknown key for a VALUE_STRING in [score_mode].", "root_cause": [ { "col": 86, "line": 14, "reason": "Unknown key for a VALUE_STRING in [score_mode].", "type": "parsing_exception" } ], "type": "parsing_exception" }, "status": 400 } ``` So, something still wrong... However, after i removed "min\_score" from query, everything started to work. Probably the problem because of wrong usage of "min\_score"? ---
**Author:** ![philross](https://avatars.discourse-cdn.com/v4/letter/p/54ee81/32.png) [@philross](https://discuss.elastic.co/u/philross)\ **Post date:** [June 21, 2019, 4:55pm UTC](https://discuss.elastic.co/t/null-pointer-exception-when-use-bool-query/186831/3 "2019-06-21T16:55:21Z")
I've unfortunately run into this same problem... the query worked fine up to and including v6.8.0, but when we updated to v7.1.1 we began experiencing the NPEs at the exact place you are. We've tried moving back to the initial GA release 7.0.0 and still experience the same problem. If we remove the min score it bypasses the error, but having the min score is pretty important for us. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [June 21, 2019, 5:09pm UTC](https://discuss.elastic.co/t/null-pointer-exception-when-use-bool-query/186831/4 "2019-06-21T17:09:01Z")
Ping @jimczi. Aware of this? ---
**Author:** ![jimczi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jimczi/32/47985_2.png) [@jimczi](https://discuss.elastic.co/u/jimczi)\ **Post date:** [June 21, 2019, 6:08pm UTC](https://discuss.elastic.co/t/null-pointer-exception-when-use-bool-query/186831/5 "2019-06-21T18:08:17Z")
The NPE is indeed a bug. Can you open an issue on github? Regarding the other error with score\_mode, function\_score and min\_score are two different features so this is expected. score\_mode is not applicable in the request body, it can only be used inside a function\_score query. ---
**Author:** ![philross](https://avatars.discourse-cdn.com/v4/letter/p/54ee81/32.png) [@philross](https://discuss.elastic.co/u/philross)\ **Post date:** [June 21, 2019, 6:27pm UTC](https://discuss.elastic.co/t/null-pointer-exception-when-use-bool-query/186831/6 "2019-06-21T18:27:14Z")
I quickly and haphazardly filed [https://github.com/elastic/elasticsearch/issues/43497](https://github.com/elastic/elasticsearch/issues/43497) My mappings and queries are probably more convoluted than what @davin4u has provided, so I'm guessing it will be easier to work with his reproduction steps. ---
**Author:** ![james.byroads](https://avatars.discourse-cdn.com/v4/letter/j/74df32/32.png) [@james.byroads](https://discuss.elastic.co/u/james.byroads)\ **Post date:** [July 12, 2019, 8:37pm UTC](https://discuss.elastic.co/t/null-pointer-exception-when-use-bool-query/186831/7 "2019-07-12T20:37:27Z")
I am also having a NPE issue, but it is different. I came across this thread when trying to find a reason/solution. Searching for "TopScoreDocCollector" is what brought me here. Figured I would piggyback off of it, but can start a new thread if more appropriate. I am on elastic v7.2.0 and can provide more details if necessary. My issue seems to be RBAC/ABAC related. As a superuser I have no issues. As a user that belongs to a role where documents must match a hard coded value, no issues. But then I have a user where {{\_user.email}} must match a particular field in the documents. For that user, when I log into Kibana, everything mostly works, but I get a "x of y shards failed" message popup. I dug a little deeper and there are five specific indices that are part of a rolling index that are the ones that fail. The DEBUG error message repeats itself five times for those five indices. Here is one of those DEBUG error messages: [I got an error message saying the body is over 7000 characters, so I am going to post DEBUG error in a second message. I used \. Is there a better way to do this??] You can see the NPE message in there. There is no mention of "MinimumScoreCollector" which I know is the context of the OPs issue. Please let me know if there are any additional details that may be of use. ---
**Author:** ![james.byroads](https://avatars.discourse-cdn.com/v4/letter/j/74df32/32.png) [@james.byroads](https://discuss.elastic.co/u/james.byroads)\ **Post date:** [July 12, 2019, 8:38pm UTC](https://discuss.elastic.co/t/null-pointer-exception-when-use-bool-query/186831/8 "2019-07-12T20:38:20Z")
``` [2019-07-10T16:46:10,508][DEBUG][o.e.a.s.TransportSearchAction] [node1] [data_2019.07.01][0], node[kYerohYBQ8-wr52Mbhvn1Q], [P], s[STARTED], a[id=dmTDB8InRR27JGPws1KNeA]: Failed to execute [SearchRequest{searchType=QUERY_THEN_FETCH, indices=[data_2018.12.26, data_2018.12.25,..., data_2019.06.25], indicesOptions=IndicesOptions[ignore_unavailable=false, allow_no_indices=true, expand_wildcards_open=true, expand_wildcards_closed=false, allow_aliases_to_multiple_indices=true, forbid_closed_indices=true, ignore_aliases=false, ignore_throttled=true], types=[], routing='null', preference='null', requestCache=false, scroll=null, maxConcurrentShardRequests=0, batchedReduceSize=512, preFilterShardSize=128, allowPartialSearchResults=true, localClusterAlias=null, getOrCreateAbsoluteStartMillis=-1, ccsMinimizeRoundtrips=true, source={}}] org.elasticsearch.transport.RemoteTransportException: [node2][10.1.1.2:9300][indices:data/read/search[phase/query]] Caused by: org.elasticsearch.search.query.QueryPhaseExecutionException: Query Failed [Failed to execute main query] at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:306) ~[elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:114) ~[elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:335) ~[elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:360) ~[elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.search.SearchService.lambda$executeQueryPhase$1(SearchService.java:340) ~[elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.action.ActionListener.lambda$map$2(ActionListener.java:145) ~[elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:62) ~[elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.search.SearchService$2.doRun(SearchService.java:1052) ~[elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:44) ~[elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:758) ~[elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.2.0.jar:7.2.0] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:834) [?:?] Caused by: java.lang.NullPointerException at org.apache.lucene.search.TopScoreDocCollector$SimpleTopScoreDocCollector$1.collect(TopScoreDocCollector.java:68) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55] at org.elasticsearch.xpack.core.security.authz.accesscontrol.SecurityIndexSearcherWrapper.intersectScorerAndRoleBits(SecurityIndexSearcherWrapper.java:199) ~[?:?] at org.elasticsearch.xpack.core.security.authz.accesscontrol.SecurityIndexSearcherWrapper$IndexSearcherWrapper.search(SecurityIndexSearcherWrapper.java:169) ~[?:?] at org.apache.lucene.search.XIndexSearcher.search(XIndexSearcher.java:44) ~[elasticsearch-7.2.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55] at org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:177) ~[elasticsearch-7.2.0.jar:7.2.0] at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:443) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55] at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:271) ~[elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:114) ~[elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:335) ~[elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:360) ~[elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.search.SearchService.lambda$executeQueryPhase$1(SearchService.java:340) ~[elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.action.ActionListener.lambda$map$2(ActionListener.java:145) ~[elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:62) ~[elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.search.SearchService$2.doRun(SearchService.java:1052) ~[elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:44) ~[elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:758) ~[elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.2.0.jar:7.2.0] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:834) ~[?:?] ``` ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [July 13, 2019, 6:12am UTC](https://discuss.elastic.co/t/null-pointer-exception-when-use-bool-query/186831/9 "2019-07-13T06:12:51Z")
Hey James Could you open an issue in GitHub with this stacktrace and if possible share the query and your settings? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 10, 2019, 6:12am UTC](https://discuss.elastic.co/t/null-pointer-exception-when-use-bool-query/186831/10 "2019-08-10T06:12:54Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.