# INTERNAL\_SERVER\_ERROR null\_pointer\_exception while percolating a document

**URL:** <https://discuss.elastic.co/t/internal-server-error-null-pointer-exception-while-percolating-a-document/99311>\
**Category:** Elasticsearch\
**Created:** [September 4, 2017, 1:50pm UTC](https://discuss.elastic.co/t/internal-server-error-null-pointer-exception-while-percolating-a-document/99311 "2017-09-04T13:50:26Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![jazz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jazz/32/6943_2.png) [@jazz](https://discuss.elastic.co/u/jazz)\
**Post date:** [September 4, 2017, 1:50pm UTC](https://discuss.elastic.co/t/internal-server-error-null-pointer-exception-while-percolating-a-document/99311/1 "2017-09-04T13:50:26Z")

</div>

We have an index with 10 shards on 10 nodes, and sometimes when a document is percolated, some nodes fail with a null\_pointer\_exception. A second try does not reproduce the exception.

```auto
status: 'INTERNAL_SERVER_ERROR',
reason: { type: 'null_pointer_exception', reason: null }

```

We use ES 2.4.1, but could not find a bug fix relating to this in later 2.4.\* releases. Did we miss it? Was it fixed in a 5.\* release?

We cannot try ES 5.\* because of incompatibilities in our code base.

I found a thread [there](https://discuss.elastic.co/t/using-percolator-causes-internal-server-error-with-npe/57778) with a similar issue but no answers.

---

<div class="post-metadata">

**Author:** ![mvg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvg/32/98890_2.png) [@mvg](https://discuss.elastic.co/u/mvg)\
**Post date:** [September 4, 2017, 2:35pm UTC](https://discuss.elastic.co/t/internal-server-error-null-pointer-exception-while-percolating-a-document/99311/2 "2017-09-04T14:35:29Z")

</div>

Hey @jazz do you see a stacktrace in your log files that related to percolating?  
If so can you share these stack traces?

I was able to reproduce the issue in the other issue that you've mentioned. Would be good to know if whether you have encountered the same error.

---

<div class="post-metadata">

**Author:** ![jazz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jazz/32/6943_2.png) [@jazz](https://discuss.elastic.co/u/jazz)\
**Post date:** [September 4, 2017, 2:48pm UTC](https://discuss.elastic.co/t/internal-server-error-null-pointer-exception-while-percolating-a-document/99311/3 "2017-09-04T14:48:36Z")

</div>

I could not see a related stack trace. Do I have to enable a setting?

---

<div class="post-metadata">

**Author:** ![mvg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvg/32/98890_2.png) [@mvg](https://discuss.elastic.co/u/mvg)\
**Post date:** [September 4, 2017, 2:52pm UTC](https://discuss.elastic.co/t/internal-server-error-null-pointer-exception-while-percolating-a-document/99311/4 "2017-09-04T14:52:14Z")

</div>

Maybe try setting the `action.percolate` to debug or trace level in `logging.yml` or update the log settings in realtime via the update cluster setting api: [https://www.elastic.co/guide/en/elasticsearch/guide/current/logging.html](https://www.elastic.co/guide/en/elasticsearch/guide/current/logging.html) ?

---

<div class="post-metadata">

**Author:** ![mvg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvg/32/98890_2.png) [@mvg](https://discuss.elastic.co/u/mvg)\
**Post date:** [September 4, 2017, 3:20pm UTC](https://discuss.elastic.co/t/internal-server-error-null-pointer-exception-while-percolating-a-document/99311/5 "2017-09-04T15:20:44Z")

</div>

@jazz I commented on the issue you mentioned with a likely cause for this issue that you're encountering.

---

<div class="post-metadata">

**Author:** ![jazz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jazz/32/6943_2.png) [@jazz](https://discuss.elastic.co/u/jazz)\
**Post date:** [September 4, 2017, 3:25pm UTC](https://discuss.elastic.co/t/internal-server-error-null-pointer-exception-while-percolating-a-document/99311/6 "2017-09-04T15:25:03Z")

</div>

With the setting in debug, I got this stacktrace:

```auto
RemoteTransportException[[es16-elasticsearch_1][172.17.0.19:9300][indices:data/read/percolate[s]]]; nested: PercolateException[failed to percolate]; nested: PercolateException[failed to execute]; nested: NullPointerException;
Caused by: PercolateException[failed to percolate]; nested: PercolateException[failed to execute]; nested: NullPointerException;
	at org.elasticsearch.action.percolate.TransportPercolateAction.shardOperation(TransportPercolateAction.java:181)
	at org.elasticsearch.action.percolate.TransportPercolateAction.shardOperation(TransportPercolateAction.java:56)
	at org.elasticsearch.action.support.broadcast.TransportBroadcastAction$ShardTransportHandler.messageReceived(TransportBroadcastAction.java:282)
	at org.elasticsearch.action.support.broadcast.TransportBroadcastAction$ShardTransportHandler.messageReceived(TransportBroadcastAction.java:278)
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:77)
	at org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.doRun(MessageChannelHandler.java:293)
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: PercolateException[failed to execute]; nested: NullPointerException;
	at org.elasticsearch.percolator.PercolatorService$5.doPercolate(PercolatorService.java:620)
	at org.elasticsearch.percolator.PercolatorService.percolate(PercolatorService.java:259)
	at org.elasticsearch.action.percolate.TransportPercolateAction.shardOperation(TransportPercolateAction.java:178)
	... 9 more
Caused by: java.lang.NullPointerException
	at org.apache.lucene.spatial.geopoint.search.GeoPointTermQueryConstantScoreWrapper$1$1.matches(GeoPointTermQueryConstantScoreWrapper.java:132)
	at org.apache.lucene.search.ConjunctionDISI$TwoPhaseConjunctionDISI.matches(ConjunctionDISI.java:248)
	at org.apache.lucene.search.ConjunctionDISI$TwoPhaseConjunctionDISI.matches(ConjunctionDISI.java:248)
	at org.apache.lucene.search.ConjunctionDISI$TwoPhase.matches(ConjunctionDISI.java:291)
	at org.apache.lucene.search.ConjunctionDISI.doNext(ConjunctionDISI.java:184)
	at org.apache.lucene.search.ConjunctionDISI.nextDoc(ConjunctionDISI.java:207)
	at org.elasticsearch.common.lucene.Lucene.exists(Lucene.java:254)
	at org.elasticsearch.percolator.QueryCollector$Match.collect(QueryCollector.java:184)
	at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:221)
	at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:172)
	at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:821)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:535)
	at org.elasticsearch.percolator.PercolatorService.queryBasedPercolating(PercolatorService.java:818)
	at org.elasticsearch.percolator.PercolatorService.access$400(PercolatorService.java:108)
	at org.elasticsearch.percolator.PercolatorService$5.doPercolate(PercolatorService.java:611)
	... 11 more

```

---

<div class="post-metadata">

**Author:** ![mvg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvg/32/98890_2.png) [@mvg](https://discuss.elastic.co/u/mvg)\
**Post date:** [September 4, 2017, 3:38pm UTC](https://discuss.elastic.co/t/internal-server-error-null-pointer-exception-while-percolating-a-document/99311/7 "2017-09-04T15:38:38Z")

</div>

Thanks for that stacktrace. This is a different failure and has to do with the fact that the in memory index that the percolator is using in 2.x doesn't support doc values (column stride storage that some geo queries use). In 5.x and beyond that percolator does support doc values.

There is a workaround that you can try to do, setting the create index version of your index to 2.1 or before as is mentioned here: [https://github.com/elastic/elasticsearch/issues/16832#issuecomment-197329289](https://github.com/elastic/elasticsearch/issues/16832#issuecomment-197329289)

---

<div class="post-metadata">

**Author:** ![jazz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jazz/32/6943_2.png) [@jazz](https://discuss.elastic.co/u/jazz)\
**Post date:** [September 4, 2017, 3:54pm UTC](https://discuss.elastic.co/t/internal-server-error-null-pointer-exception-while-percolating-a-document/99311/8 "2017-09-04T15:54:34Z")

</div>

> [@mvg](#):
>
> There is a workaround that you can try to do, setting the create index version of your index to 2.1 or before as is mentioned here: [Percolation with Bounding Box GeoLocation throws NullPointerException in Lucene · Issue #16832 · elastic/elasticsearch · GitHub](https://github.com/elastic/elasticsearch/issues/16832#issuecomment-197329289)

Do I have to re-create the index with the version set to 2.1 or can I change the version of the index?

---

<div class="post-metadata">

**Author:** ![jazz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jazz/32/6943_2.png) [@jazz](https://discuss.elastic.co/u/jazz)\
**Post date:** [September 4, 2017, 4:23pm UTC](https://discuss.elastic.co/t/internal-server-error-null-pointer-exception-while-percolating-a-document/99311/9 "2017-09-04T16:23:09Z")

</div>

Hum, we have many regular geo queries on the index that hold the percolators, it does not seem to be a good idea to change the version of this index performance-wise.

It seems like we have two alternatives:

- migrate to ES 5.x
- create a second index with same documents as the first one, but with the index version set to 2.1, where the percolators will be stored. Not sure if it will good for our cluster with twice the data.

---

<div class="post-metadata">

**Author:** ![mvg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvg/32/98890_2.png) [@mvg](https://discuss.elastic.co/u/mvg)\
**Post date:** [September 5, 2017, 4:38am UTC](https://discuss.elastic.co/t/internal-server-error-null-pointer-exception-while-percolating-a-document/99311/10 "2017-09-05T04:38:41Z")

</div>

> Do I have to re-create the index with the version set to 2.1 or can I change the version of the index?

What you can try is close the index and then adding the index created settings via the update index settings api. I think it works, but I'm not 100% sure.

> Hum, we have many regular geo queries on the index that hold the percolators, it does not seem to be a good idea to change the version of this index performance-wise.

These improvements added in later 2.x versions are only applicable if these geo queries run on a real index. Not in the case of the percolator, because the geo queries are executed on a in memory index holding a single document.

---

<div class="post-metadata">

**Author:** ![jazz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jazz/32/6943_2.png) [@jazz](https://discuss.elastic.co/u/jazz)\
**Post date:** [September 5, 2017, 9:59am UTC](https://discuss.elastic.co/t/internal-server-error-null-pointer-exception-while-percolating-a-document/99311/11 "2017-09-05T09:59:22Z")

</div>

> [@mvg](#):
>
> These improvements added in later 2.x versions are only applicable if these geo queries run on a real index. Not in the case of the percolator, because the geo queries are executed on a in memory index holding a single document.

The issue is that we have lots of geo queries on this index beside the percolators.

---

<div class="post-metadata">

**Author:** ![mvg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvg/32/98890_2.png) [@mvg](https://discuss.elastic.co/u/mvg)\
**Post date:** [September 5, 2017, 10:49am UTC](https://discuss.elastic.co/t/internal-server-error-null-pointer-exception-while-percolating-a-document/99311/12 "2017-09-05T10:49:37Z")

</div>

> The issue is that we have lots of geo queries on this index beside the percolators.

Then maybe split the percolator queries from the real data in a dedicated index? If you do that then make sure that you that you also copy the mapping for the documents you percolate into the dedicated index.

---

<div class="post-metadata">

**Author:** ![jazz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jazz/32/6943_2.png) [@jazz](https://discuss.elastic.co/u/jazz)\
**Post date:** [September 5, 2017, 11:31am UTC](https://discuss.elastic.co/t/internal-server-error-null-pointer-exception-while-percolating-a-document/99311/13 "2017-09-05T11:31:01Z")

</div>

I missed the fact that I can have the percolator queries in a dedicated index! I will try this, thanks for your help!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 3, 2017, 11:31am UTC](https://discuss.elastic.co/t/internal-server-error-null-pointer-exception-while-percolating-a-document/99311/14 "2017-10-03T11:31:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.