Null pointer exception for explain with boost mode replace

Elastic Stack Elasticsearch
1

We seem to have odd situation for requesting explanation when boost mode is set to replace, the stack returns null pointer exception. Running on Version: 8.7.1, Build: docker/f229ed3f893a515d590d0f39b05f68913e2d9b53/2023-04-27T04:33:42.127815583Z, JVM: 20.0.1

Here is the minimal request that causes the error (actual request is a bit more involved):

{
  "query": {
    "bool": {
      "filter": [],
      "must": [
        {
          "function_score": {
            "functions": [
              {
                "filter": {
                  "simple_query_string": {
                    "fields": [
                      "cv_text"
                    ],
                    "query": "\"Java\"|((\"SAP\"|\"BW\")|\"SAP BW\")",
                    "_name": "cv search"
                  }
                },
                "weight": 1
              }
            ],
            "query": {
              "simple_query_string": {
                "fields": [
                  "terms_text^5",
                  "terms_plain_text^3",
                  "cv_text^2"
                ],
                "query": "\"Java\"|((\"SAP\"|\"BW\")|\"SAP BW\")"
              }
            },
            "boost_mode": "replace",
            "score_mode": "sum"
          }
        }
      ]
    }
  },
  "explain": true,
  "_source": false
}

With explain: false the request succeeds, otherwise it results in below error. Any ideas what may be wrong?

Failed to execute phase [query], all shards failed; shardFailures {[fEvS1WFnRTW-5pt1fxX3nQ][candidates][0]: org.elasticsearch.transport.RemoteTransportException: [quickstart-es-default-0][10.244.2.38:9300][indices:data/read/search[phase/query]]
Caused by: org.elasticsearch.search.fetch.FetchPhaseExecutionException: Fetch Failed [Error running fetch phase for doc [3]]
	at org.elasticsearch.server@8.7.1/org.elasticsearch.search.fetch.FetchPhaseDocsIterator.iterate(FetchPhaseDocsIterator.java:73)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.search.fetch.FetchPhase.buildSearchHits(FetchPhase.java:161)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:76)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:672)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:644)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.search.SearchService.lambda$executeQueryPhase$2(SearchService.java:495)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.action.ActionRunnable$2.accept(ActionRunnable.java:50)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.action.ActionRunnable$2.accept(ActionRunnable.java:47)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.action.ActionRunnable$3.doRun(ActionRunnable.java:72)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:33)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:958)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1623)
Caused by: java.lang.NullPointerException: Cannot invoke "org.apache.lucene.search.LeafSimScorer.score(int, float)" because "this.docScorer" is null
	at org.apache.lucene.queries@9.5.0/org.apache.lucene.queries.spans.SpanScorer.scoreCurrentDoc(SpanScorer.java:74)
	at org.apache.lucene.queries@9.5.0/org.apache.lucene.queries.spans.SpanScorer.score(SpanScorer.java:131)
	at org.apache.lucene.core@9.5.0/org.apache.lucene.search.DisjunctionMaxScorer.score(DisjunctionMaxScorer.java:65)
	at org.apache.lucene.core@9.5.0/org.apache.lucene.search.DisjunctionScorer.score(DisjunctionScorer.java:193)
	at org.apache.lucene.core@9.5.0/org.apache.lucene.search.ConjunctionScorer.score(ConjunctionScorer.java:61)
	at org.apache.lucene.core@9.5.0/org.apache.lucene.search.BooleanWeight.explain(BooleanWeight.java:122)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.common.lucene.search.function.FunctionScoreQuery$CustomBoostFactorWeight.explain(FunctionScoreQuery.java:305)
	at org.apache.lucene.core@9.5.0/org.apache.lucene.search.BooleanWeight.explain(BooleanWeight.java:75)
	at org.apache.lucene.core@9.5.0/org.apache.lucene.search.IndexSearcher.explain(IndexSearcher.java:893)
	at org.apache.lucene.core@9.5.0/org.apache.lucene.search.IndexSearcher.explain(IndexSearcher.java:869)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.search.fetch.subphase.ExplainPhase$1.process(ExplainPhase.java:38)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.search.fetch.FetchPhase$1.nextDoc(FetchPhase.java:155)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.search.fetch.FetchPhaseDocsIterator.iterate(FetchPhaseDocsIterator.java:70)
	... 15 more
}
	at org.elasticsearch.server@8.7.1/org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:728)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:418)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:760)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:512)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.action.search.AbstractSearchAsyncAction$1.onFailure(AbstractSearchAsyncAction.java:349)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.action.ActionListener$Delegating.onFailure(ActionListener.java:97)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:48)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.action.search.SearchTransportService$ConnectionCountingHandler.handleException(SearchTransportService.java:644)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.transport.TransportService$UnregisterChildTransportResponseHandler.handleException(TransportService.java:1651)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1377)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.transport.TransportService$DirectResponseChannel.processException(TransportService.java:1513)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1488)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.transport.TaskTransportChannel.sendResponse(TaskTransportChannel.java:50)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.action.support.ChannelActionListener.onFailure(ChannelActionListener.java:44)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.action.ActionRunnable.onFailure(ActionRunnable.java:92)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:28)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:33)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:958)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1623)
Caused by: org.elasticsearch.ElasticsearchException$1: Cannot invoke "org.apache.lucene.search.LeafSimScorer.score(int, float)" because "this.docScorer" is null
	at org.elasticsearch.server@8.7.1/org.elasticsearch.ElasticsearchException.guessRootCauses(ElasticsearchException.java:668)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:416)
	... 20 more
Caused by: java.lang.NullPointerException: Cannot invoke "org.apache.lucene.search.LeafSimScorer.score(int, float)" because "this.docScorer" is null
	at org.apache.lucene.queries@9.5.0/org.apache.lucene.queries.spans.SpanScorer.scoreCurrentDoc(SpanScorer.java:74)
	at org.apache.lucene.queries@9.5.0/org.apache.lucene.queries.spans.SpanScorer.score(SpanScorer.java:131)
	at org.apache.lucene.core@9.5.0/org.apache.lucene.search.DisjunctionMaxScorer.score(DisjunctionMaxScorer.java:65)
	at org.apache.lucene.core@9.5.0/org.apache.lucene.search.DisjunctionScorer.score(DisjunctionScorer.java:193)
	at org.apache.lucene.core@9.5.0/org.apache.lucene.search.ConjunctionScorer.score(ConjunctionScorer.java:61)
	at org.apache.lucene.core@9.5.0/org.apache.lucene.search.BooleanWeight.explain(BooleanWeight.java:122)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.common.lucene.search.function.FunctionScoreQuery$CustomBoostFactorWeight.explain(FunctionScoreQuery.java:305)
	at org.apache.lucene.core@9.5.0/org.apache.lucene.search.BooleanWeight.explain(BooleanWeight.java:75)
	at org.apache.lucene.core@9.5.0/org.apache.lucene.search.IndexSearcher.explain(IndexSearcher.java:893)
	at org.apache.lucene.core@9.5.0/org.apache.lucene.search.IndexSearcher.explain(IndexSearcher.java:869)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.search.fetch.subphase.ExplainPhase$1.process(ExplainPhase.java:38)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.search.fetch.FetchPhase$1.nextDoc(FetchPhase.java:155)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.search.fetch.FetchPhaseDocsIterator.iterate(FetchPhaseDocsIterator.java:70)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.search.fetch.FetchPhase.buildSearchHits(FetchPhase.java:161)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:76)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:672)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:644)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.search.SearchService.lambda$executeQueryPhase$2(SearchService.java:495)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.action.ActionRunnable$2.accept(ActionRunnable.java:50)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.action.ActionRunnable$2.accept(ActionRunnable.java:47)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.action.ActionRunnable$3.doRun(ActionRunnable.java:72)
	at org.elasticsearch.server@8.7.1/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)
	... 6 more
2

One thing I found is that usage of quoted single words seems to cause the issue. E.g. "Java|((SAP|BW)|\"SAP BW\")" vs "\"Java\"|((\"SAP\"|\"BW\")|\"SAP BW\")". So the below query actually does not break, and that is the only change.

{
  "query": {
    "bool": {
      "filter": [],
      "must": [
        {
          "function_score": {
            "functions": [
              {
                "filter": {
                  "simple_query_string": {
                    "fields": [
                      "cv_text"
                    ],
                    "query": "Java|((SAP|BW)|\"SAP BW\")",
                    "_name": "cv search"
                  }
                },
                "weight": 1
              }
            ],
            "query": {
              "simple_query_string": {
                "fields": [
                  "terms_text^5",
                  "terms_plain_text^3",
                  "cv_text^2"
                ],
                "query": "Java|((SAP|BW)|\"SAP BW\")"
              }
            },
            "boost_mode": "replace",
            "score_mode": "sum"
          }
        }
      ]
    }
  },
  "explain": true,
  "_source": false
}
3

However, for a more elaborate partially quoted query like "((Senior+Java+Developer)|\"Senior Java Developer\")+(((SAP|BW)|\"SAP BW\")|Ruby)" we still have the nullpointer exception.

When I remove quotes completely, as in "((Senior+Java+Developer)|Senior Java Developer)+(((SAP|BW)|SAP BW)|Ruby)", then again there are no errors. But, it returns completely different results so it is not suitable.

Funny enough, "((Senior+Java+Developer)|\"Senior Java Developer\")+(((SAP+BW)|\"SAP BW\")|Ruby)" poses no trouble.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.