# \[MergeException\[java.lang.NullPointerException\] and All shards failed for phase: \[query\_fetch\] **URL:** \ **Category:** Elasticsearch\ **Created:** [February 6, 2015, 12:01am UTC](https://discuss.elastic.co/t/mergeexception-java-lang-nullpointerexception-and-all-shards-failed-for-phase-query-fetch/22005 "2015-02-06T00:01:09Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![ElasticSearch\_Users\_](https://avatars.discourse-cdn.com/v4/letter/e/b4bc9f/32.png) [@ElasticSearch\_Users\_](https://discuss.elastic.co/u/ElasticSearch_Users_)\ **Post date:** [February 6, 2015, 12:01am UTC](https://discuss.elastic.co/t/mergeexception-java-lang-nullpointerexception-and-all-shards-failed-for-phase-query-fetch/22005/1 "2015-02-06T00:01:09Z")
Hello, My environment has 1 linux server installed elasticsearch 1.4.2 with default settings from rpm package. I use TransportClient to send requests to elasticsearch. I recently changed to use rpm package. It has been working fine for a couple of day. But today I saw the following errors when I indexed a data set I had successfully indexed before and tried to query a simple word. In the meantime, the server takes much longer time to index and delete indices. [2015-02-05 18:01:01,105][WARN][index.merge.scheduler] [Kkallakku] [wa\_value\_index\_v1][0] failed to merge java.lang.NullPointerException at org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer$NormMap.getOrd(Lucene49NormsConsumer.java:249) at org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer.addNumericField(Lucene49NormsConsumer.java:150) at org.apache.lucene.codecs.DocValuesConsumer.mergeNumericField(DocValuesConsumer.java:129) at org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:255) at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:133) at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4173) at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3768) at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405) at org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107) at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482) [2015-02-05 18:01:01,106][WARN][index.engine.internal] [Kkallakku] [wa\_value\_index\_v1][0] failed engine [merge exception] org.apache.lucene.index.MergePolicy$MergeException: java.lang.NullPointerException at org.elasticsearch.index.merge.scheduler.ConcurrentMergeSchedulerProvider$CustomConcurrentMergeScheduler.handleMergeException(ConcurrentMergeSchedulerProvider.java:133) at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:518) Caused by: java.lang.NullPointerException at org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer$NormMap.getOrd(Lucene49NormsConsumer.java:249) at org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer.addNumericField(Lucene49NormsConsumer.java:150) at org.apache.lucene.codecs.DocValuesConsumer.mergeNumericField(DocValuesConsumer.java:129) at org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:255) at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:133) at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4173) at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3768) at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405) at org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107) at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482) [2015-02-05 18:01:01,369][WARN][cluster.action.shard] [Kkallakku] [wa\_value\_index\_v1][0] sending failed shard for [wa\_value\_index\_v1][0], node[KpdH3su1QSyTx8lekwshAA], [P], s[STARTED], indexUUID [LgwNpUo7RRqjnIaVcA2rLA], reason [engine failure, message [merge exception][MergeException[java.lang.NullPointerException]; nested: NullPointerException; ]] [2015-02-05 18:01:01,369][WARN][cluster.action.shard] [Kkallakku] [wa\_value\_index\_v1][0] received shard failed for [wa\_value\_index\_v1][0], node[KpdH3su1QSyTx8lekwshAA], [P], s[STARTED], indexUUID [LgwNpUo7RRqjnIaVcA2rLA], reason [engine failure, message [merge exception][MergeException[java.lang.NullPointerException]; nested: NullPointerException; ]] org.elasticsearch.action.search.SearchPhaseExecutionException: Failed to execute phase [query\_fetch], all shards failed at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:233) at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.start(TransportSearchTypeAction.java:156) at org.elasticsearch.action.search.type.TransportSearchQueryAndFetchAction.doExecute(TransportSearchQueryAndFetchAction.java:55) at org.elasticsearch.action.search.type.TransportSearchQueryAndFetchAction.doExecute(TransportSearchQueryAndFetchAction.java:45) at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:75) at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:107) at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:45) Has any one seen these issues before? What are the causes? Many thanks, Cindy -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/637890eb-d50f-4932-a511-a3f7fb3f6aa9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/637890eb-d50f-4932-a511-a3f7fb3f6aa9%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Robert\_Muir\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/robert_muir_2/32/931_2.png) [@Robert\_Muir\_2](https://discuss.elastic.co/u/Robert_Muir_2)\ **Post date:** [February 6, 2015, 12:25am UTC](https://discuss.elastic.co/t/mergeexception-java-lang-nullpointerexception-and-all-shards-failed-for-phase-query-fetch/22005/2 "2015-02-06T00:25:21Z")
Are you using an IBM JDK? Don't do that 🙂 On Thu, Feb 5, 2015 at 7:01 PM, 'Cindy' via elasticsearch [elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com) wrote: > Hello, > > My environment has 1 linux server installed elasticsearch 1.4.2 with default > settings from rpm package. I use TransportClient to send requests to > elasticsearch. I recently changed to use rpm package. It has been working > fine for a couple of day. But today I saw the following errors when I > indexed a data set I had successfully indexed before and tried to query a > simple word. In the meantime, the server takes much longer time to index and > delete indices. > > [2015-02-05 18:01:01,105][WARN][index.merge.scheduler] [Kkallakku] > [wa\_value\_index\_v1][0] failed to merge > java.lang.NullPointerException > at > org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer$NormMap.getOrd(Lucene49NormsConsumer.java:249) > at > org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer.addNumericField(Lucene49NormsConsumer.java:150) > at > org.apache.lucene.codecs.DocValuesConsumer.mergeNumericField(DocValuesConsumer.java:129) > at > org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:255) > at > org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:133) > at > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4173) > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3768) > at > org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405) > at > org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107) > at > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482) > [2015-02-05 18:01:01,106][WARN][index.engine.internal] [Kkallakku] > [wa\_value\_index\_v1][0] failed engine [merge exception] > org.apache.lucene.index.MergePolicy$MergeException: > java.lang.NullPointerException > at > org.elasticsearch.index.merge.scheduler.ConcurrentMergeSchedulerProvider$CustomConcurrentMergeScheduler.handleMergeException(ConcurrentMergeSchedulerProvider.java:133) > at > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:518) > Caused by: java.lang.NullPointerException > at > org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer$NormMap.getOrd(Lucene49NormsConsumer.java:249) > at > org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer.addNumericField(Lucene49NormsConsumer.java:150) > at > org.apache.lucene.codecs.DocValuesConsumer.mergeNumericField(DocValuesConsumer.java:129) > at > org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:255) > at > org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:133) > at > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4173) > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3768) > at > org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405) > at > org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107) > at > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482) > [2015-02-05 18:01:01,369][WARN][cluster.action.shard] [Kkallakku] > [wa\_value\_index\_v1][0] sending failed shard for [wa\_value\_index\_v1][0], > node[KpdH3su1QSyTx8lekwshAA], [P], s[STARTED], indexUUID > [LgwNpUo7RRqjnIaVcA2rLA], reason [engine failure, message [merge > exception][MergeException[java.lang.NullPointerException]; nested: > NullPointerException; ]] > [2015-02-05 18:01:01,369][WARN][cluster.action.shard] [Kkallakku] > [wa\_value\_index\_v1][0] received shard failed for [wa\_value\_index\_v1][0], > node[KpdH3su1QSyTx8lekwshAA], [P], s[STARTED], indexUUID > [LgwNpUo7RRqjnIaVcA2rLA], reason [engine failure, message [merge > exception][MergeException[java.lang.NullPointerException]; nested: > NullPointerException; ]] > > org.elasticsearch.action.search.SearchPhaseExecutionException: Failed to > execute phase [query\_fetch], all shards failed > at > org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:233) > at > org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.start(TransportSearchTypeAction.java:156) > at > org.elasticsearch.action.search.type.TransportSearchQueryAndFetchAction.doExecute(TransportSearchQueryAndFetchAction.java:55) > at > org.elasticsearch.action.search.type.TransportSearchQueryAndFetchAction.doExecute(TransportSearchQueryAndFetchAction.java:45) > at > org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:75) > at > org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:107) > at > org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:45) > > Has any one seen these issues before? What are the causes? > > Many thanks, > Cindy > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/637890eb-d50f-4932-a511-a3f7fb3f6aa9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/637890eb-d50f-4932-a511-a3f7fb3f6aa9%40googlegroups.com). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAMUKNZVROAeQjM1UKCgdxXs52S81Yaad74%3D4kDdpeOXarNf1aA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAMUKNZVROAeQjM1UKCgdxXs52S81Yaad74%3D4kDdpeOXarNf1aA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![ElasticSearch\_Users\_](https://avatars.discourse-cdn.com/v4/letter/e/b4bc9f/32.png) [@ElasticSearch\_Users\_](https://discuss.elastic.co/u/ElasticSearch_Users_)\ **Post date:** [February 6, 2015, 12:39am UTC](https://discuss.elastic.co/t/mergeexception-java-lang-nullpointerexception-and-all-shards-failed-for-phase-query-fetch/22005/3 "2015-02-06T00:39:47Z")
Yes, I am using IBM JDK. How do you detect it that? Does it show from the log? I know there is a Lucene IBM JVM bug. Is this related? What are the cause of the above exceptions? Thank you, Cindy On Thursday, 5 February 2015 19:25:49 UTC-5, Robert Muir wrote: > Are you using an IBM JDK? Don't do that 🙂 > > On Thu, Feb 5, 2015 at 7:01 PM, 'Cindy' via elasticsearch > \<[elasti...@googlegroups.com](mailto:elasti...@googlegroups.com) \\> wrote: > > > Hello, > > > > My environment has 1 linux server installed elasticsearch 1.4.2 with > > default > > settings from rpm package. I use TransportClient to send requests to > > elasticsearch. I recently changed to use rpm package. It has been > > working > > fine for a couple of day. But today I saw the following errors when I > > indexed a data set I had successfully indexed before and tried to query > > a > > simple word. In the meantime, the server takes much longer time to index > > and > > delete indices. > > > > [2015-02-05 18:01:01,105][WARN][index.merge.scheduler] [Kkallakku] > > [wa\_value\_index\_v1][0] failed to merge > > java.lang.NullPointerException > > at > > org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer$NormMap.getOrd(Lucene49NormsConsumer.java:249) > > > ``` > > at > > > > ``` > > org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer.addNumericField(Lucene49NormsConsumer.java:150) > > > ``` > > at > > > > ``` > > org.apache.lucene.codecs.DocValuesConsumer.mergeNumericField(DocValuesConsumer.java:129) > > > ``` > > at > > > > ``` > > > > org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:255) > > at > > org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:133) > > at > > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4173) > > at > > org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3768) > > at > > org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405) > > > ``` > > at > > > > ``` > > org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107) > > > ``` > > at > > > > ``` > > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482) > > > [2015-02-05 18:01:01,106][WARN][index.engine.internal] [Kkallakku] > > [wa\_value\_index\_v1][0] failed engine [merge exception] > > org.apache.lucene.index.MergePolicy$MergeException: > > java.lang.NullPointerException > > at > > org.elasticsearch.index.merge.scheduler.ConcurrentMergeSchedulerProvider$CustomConcurrentMergeScheduler.handleMergeException(ConcurrentMergeSchedulerProvider.java:133) > > > ``` > > at > > > > ``` > > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:518) > > > Caused by: java.lang.NullPointerException > > at > > org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer$NormMap.getOrd(Lucene49NormsConsumer.java:249) > > > ``` > > at > > > > ``` > > org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer.addNumericField(Lucene49NormsConsumer.java:150) > > > ``` > > at > > > > ``` > > org.apache.lucene.codecs.DocValuesConsumer.mergeNumericField(DocValuesConsumer.java:129) > > > ``` > > at > > > > ``` > > > > org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:255) > > at > > org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:133) > > at > > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4173) > > at > > org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3768) > > at > > org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405) > > > ``` > > at > > > > ``` > > org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107) > > > ``` > > at > > > > ``` > > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482) > > > [2015-02-05 18:01:01,369][WARN][cluster.action.shard] [Kkallakku] > > [wa\_value\_index\_v1][0] sending failed shard for [wa\_value\_index\_v1][0], > > node[KpdH3su1QSyTx8lekwshAA], [P], s[STARTED], indexUUID > > [LgwNpUo7RRqjnIaVcA2rLA], reason [engine failure, message [merge > > exception][MergeException[java.lang.NullPointerException]; nested: > > NullPointerException; ]] > > [2015-02-05 18:01:01,369][WARN][cluster.action.shard] [Kkallakku] > > [wa\_value\_index\_v1][0] received shard failed for [wa\_value\_index\_v1][0], > > node[KpdH3su1QSyTx8lekwshAA], [P], s[STARTED], indexUUID > > [LgwNpUo7RRqjnIaVcA2rLA], reason [engine failure, message [merge > > exception][MergeException[java.lang.NullPointerException]; nested: > > NullPointerException; ]] > > > > org.elasticsearch.action.search.SearchPhaseExecutionException: Failed to > > execute phase [query\_fetch], all shards failed > > at > > org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:233) > > > at > > org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.start(TransportSearchTypeAction.java:156) > > > at > > org.elasticsearch.action.search.type.TransportSearchQueryAndFetchAction.doExecute(TransportSearchQueryAndFetchAction.java:55) > > > at > > org.elasticsearch.action.search.type.TransportSearchQueryAndFetchAction.doExecute(TransportSearchQueryAndFetchAction.java:45) > > > at > > org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:75) > > > at > > org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:107) > > > at > > org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:45) > > > Has any one seen these issues before? What are the causes? > > > > Many thanks, > > Cindy > > > > -- > > You received this message because you are subscribed to the Google > > Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/637890eb-d50f-4932-a511-a3f7fb3f6aa9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/637890eb-d50f-4932-a511-a3f7fb3f6aa9%40googlegroups.com). > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/31187132-7efd-4fa4-b31b-1e008f6933d6%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/31187132-7efd-4fa4-b31b-1e008f6933d6%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Robert\_Muir\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/robert_muir_2/32/931_2.png) [@Robert\_Muir\_2](https://discuss.elastic.co/u/Robert_Muir_2)\ **Post date:** [February 6, 2015, 12:43am UTC](https://discuss.elastic.co/t/mergeexception-java-lang-nullpointerexception-and-all-shards-failed-for-phase-query-fetch/22005/4 "2015-02-06T00:43:59Z")
I know this because I have seen this same IBM JDK bug in our tests many times. I also wrote the code in question. NPE is not possible. [https://github.com/apache/lucene-solr/blob/lucene\_solr\_4\_10\_2/lucene/core/src/java/org/apache/lucene/codecs/lucene49/Lucene49NormsConsumer.java#L213](https://github.com/apache/lucene-solr/blob/lucene_solr_4_10_2/lucene/core/src/java/org/apache/lucene/codecs/lucene49/Lucene49NormsConsumer.java#L213) This is why we say, don't use IBM JDK with lucene. On Thu, Feb 5, 2015 at 7:39 PM, 'Cindy' via elasticsearch [elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com) wrote: > Yes, I am using IBM JDK. How do you detect it that? Does it show from the > log? > > I know there is a Lucene IBM JVM bug. Is this related? What are the cause of > the above exceptions? > > Thank you, > > Cindy > > On Thursday, 5 February 2015 19:25:49 UTC-5, Robert Muir wrote: > > > Are you using an IBM JDK? Don't do that 🙂 > > > > On Thu, Feb 5, 2015 at 7:01 PM, 'Cindy' via elasticsearch > > [elasti...@googlegroups.com](mailto:elasti...@googlegroups.com) wrote: > > > > > Hello, > > > > > > My environment has 1 linux server installed elasticsearch 1.4.2 with > > > default > > > settings from rpm package. I use TransportClient to send requests to > > > elasticsearch. I recently changed to use rpm package. It has been > > > working > > > fine for a couple of day. But today I saw the following errors when I > > > indexed a data set I had successfully indexed before and tried to query > > > a > > > simple word. In the meantime, the server takes much longer time to index > > > and > > > delete indices. > > > > > > [2015-02-05 18:01:01,105][WARN][index.merge.scheduler] [Kkallakku] > > > [wa\_value\_index\_v1][0] failed to merge > > > java.lang.NullPointerException > > > at > > > > > > org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer$NormMap.getOrd(Lucene49NormsConsumer.java:249) > > > at > > > > > > org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer.addNumericField(Lucene49NormsConsumer.java:150) > > > at > > > > > > org.apache.lucene.codecs.DocValuesConsumer.mergeNumericField(DocValuesConsumer.java:129) > > > at > > > org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:255) > > > at > > > org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:133) > > > at > > > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4173) > > > at > > > org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3768) > > > at > > > > > > org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405) > > > at > > > > > > org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107) > > > at > > > > > > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482) > > > [2015-02-05 18:01:01,106][WARN][index.engine.internal] [Kkallakku] > > > [wa\_value\_index\_v1][0] failed engine [merge exception] > > > org.apache.lucene.index.MergePolicy$MergeException: > > > java.lang.NullPointerException > > > at > > > > > > org.elasticsearch.index.merge.scheduler.ConcurrentMergeSchedulerProvider$CustomConcurrentMergeScheduler.handleMergeException(ConcurrentMergeSchedulerProvider.java:133) > > > at > > > > > > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:518) > > > Caused by: java.lang.NullPointerException > > > at > > > > > > org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer$NormMap.getOrd(Lucene49NormsConsumer.java:249) > > > at > > > > > > org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer.addNumericField(Lucene49NormsConsumer.java:150) > > > at > > > > > > org.apache.lucene.codecs.DocValuesConsumer.mergeNumericField(DocValuesConsumer.java:129) > > > at > > > org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:255) > > > at > > > org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:133) > > > at > > > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4173) > > > at > > > org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3768) > > > at > > > > > > org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405) > > > at > > > > > > org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107) > > > at > > > > > > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482) > > > [2015-02-05 18:01:01,369][WARN][cluster.action.shard] [Kkallakku] > > > [wa\_value\_index\_v1][0] sending failed shard for [wa\_value\_index\_v1][0], > > > node[KpdH3su1QSyTx8lekwshAA], [P], s[STARTED], indexUUID > > > [LgwNpUo7RRqjnIaVcA2rLA], reason [engine failure, message [merge > > > exception][MergeException[java.lang.NullPointerException]; nested: > > > NullPointerException; ]] > > > [2015-02-05 18:01:01,369][WARN][cluster.action.shard] [Kkallakku] > > > [wa\_value\_index\_v1][0] received shard failed for [wa\_value\_index\_v1][0], > > > node[KpdH3su1QSyTx8lekwshAA], [P], s[STARTED], indexUUID > > > [LgwNpUo7RRqjnIaVcA2rLA], reason [engine failure, message [merge > > > exception][MergeException[java.lang.NullPointerException]; nested: > > > NullPointerException; ]] > > > > > > org.elasticsearch.action.search.SearchPhaseExecutionException: Failed to > > > execute phase [query\_fetch], all shards failed > > > at > > > > > > org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:233) > > > at > > > > > > org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.start(TransportSearchTypeAction.java:156) > > > at > > > > > > org.elasticsearch.action.search.type.TransportSearchQueryAndFetchAction.doExecute(TransportSearchQueryAndFetchAction.java:55) > > > at > > > > > > org.elasticsearch.action.search.type.TransportSearchQueryAndFetchAction.doExecute(TransportSearchQueryAndFetchAction.java:45) > > > at > > > > > > org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:75) > > > at > > > > > > org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:107) > > > at > > > > > > org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:45) > > > > > > Has any one seen these issues before? What are the causes? > > > > > > Many thanks, > > > Cindy > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups > > > "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an > > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > To view this discussion on the web visit > > > > > > [https://groups.google.com/d/msgid/elasticsearch/637890eb-d50f-4932-a511-a3f7fb3f6aa9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/637890eb-d50f-4932-a511-a3f7fb3f6aa9%40googlegroups.com). > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/31187132-7efd-4fa4-b31b-1e008f6933d6%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/31187132-7efd-4fa4-b31b-1e008f6933d6%40googlegroups.com). > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAMUKNZX%2BZLRNW8KYwq0mERj%3DfFAv34f6PeOazd36AaOQ-pSmaA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAMUKNZX%2BZLRNW8KYwq0mERj%3DfFAv34f6PeOazd36AaOQ-pSmaA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![ElasticSearch\_Users\_](https://avatars.discourse-cdn.com/v4/letter/e/b4bc9f/32.png) [@ElasticSearch\_Users\_](https://discuss.elastic.co/u/ElasticSearch_Users_)\ **Post date:** [February 6, 2015, 12:53am UTC](https://discuss.elastic.co/t/mergeexception-java-lang-nullpointerexception-and-all-shards-failed-for-phase-query-fetch/22005/5 "2015-02-06T00:53:12Z")
It's really appreciated that you quickly pointed out the cause!!! Actually I had posted a question in this forum trying to find out more details about the IBM JVM bug and why I can't use IBM JVM. Now I knew it 🙂 On Thursday, 5 February 2015 19:44:26 UTC-5, Robert Muir wrote: > I know this because I have seen this same IBM JDK bug in our tests many > times. > > I also wrote the code in question. NPE is not possible. > > [https://github.com/apache/lucene-solr/blob/lucene\_solr\_4\_10\_2/lucene/core/src/java/org/apache/lucene/codecs/lucene49/Lucene49NormsConsumer.java#L213](https://github.com/apache/lucene-solr/blob/lucene_solr_4_10_2/lucene/core/src/java/org/apache/lucene/codecs/lucene49/Lucene49NormsConsumer.java#L213) > > This is why we say, don't use IBM JDK with lucene. > > On Thu, Feb 5, 2015 at 7:39 PM, 'Cindy' via elasticsearch > \<[elasti...@googlegroups.com](mailto:elasti...@googlegroups.com) \\> wrote: > > > Yes, I am using IBM JDK. How do you detect it that? Does it show from > > the > > log? > > > > I know there is a Lucene IBM JVM bug. Is this related? What are the > > cause of > > the above exceptions? > > > > Thank you, > > > > Cindy > > > > On Thursday, 5 February 2015 19:25:49 UTC-5, Robert Muir wrote: > > > > > Are you using an IBM JDK? Don't do that 🙂 > > > > > > On Thu, Feb 5, 2015 at 7:01 PM, 'Cindy' via elasticsearch > > > [elasti...@googlegroups.com](mailto:elasti...@googlegroups.com) wrote: > > > > > > > Hello, > > > > > > > > My environment has 1 linux server installed elasticsearch 1.4.2 with > > > > default > > > > settings from rpm package. I use TransportClient to send requests to > > > > elasticsearch. I recently changed to use rpm package. It has been > > > > working > > > > fine for a couple of day. But today I saw the following errors when I > > > > indexed a data set I had successfully indexed before and tried to > > > > query > > > > a > > > > simple word. In the meantime, the server takes much longer time to > > > > index > > > > and > > > > delete indices. > > > > > > > > [2015-02-05 18:01:01,105][WARN][index.merge.scheduler] > > > > [Kkallakku] > > > > [wa\_value\_index\_v1][0] failed to merge > > > > java.lang.NullPointerException > > > > at > > org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer$NormMap.getOrd(Lucene49NormsConsumer.java:249) > > > > > ``` > > > > at > > > > > > > > ``` > > org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer.addNumericField(Lucene49NormsConsumer.java:150) > > > > > ``` > > > > at > > > > > > > > ``` > > org.apache.lucene.codecs.DocValuesConsumer.mergeNumericField(DocValuesConsumer.java:129) > > > > > ``` > > > > at > > > > > > > > ``` > > org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:255) > > > > > ``` > > > > at > > > > > > > > ``` > > > > > > > > org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:133) > > > > at > > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4173) > > > > > ``` > > > > at > > > > > > > > ``` > > > > > > > > org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3768) > > > > at > > org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405) > > > > > ``` > > > > at > > > > > > > > ``` > > org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107) > > > > > ``` > > > > at > > > > > > > > ``` > > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482) > > > > > [2015-02-05 18:01:01,106][WARN][index.engine.internal] > > > > [Kkallakku] > > > > [wa\_value\_index\_v1][0] failed engine [merge exception] > > > > org.apache.lucene.index.MergePolicy$MergeException: > > > > java.lang.NullPointerException > > > > at > > org.elasticsearch.index.merge.scheduler.ConcurrentMergeSchedulerProvider$CustomConcurrentMergeScheduler.handleMergeException(ConcurrentMergeSchedulerProvider.java:133) > > > > > ``` > > > > at > > > > > > > > ``` > > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:518) > > > > > Caused by: java.lang.NullPointerException > > > > at > > org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer$NormMap.getOrd(Lucene49NormsConsumer.java:249) > > > > > ``` > > > > at > > > > > > > > ``` > > org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer.addNumericField(Lucene49NormsConsumer.java:150) > > > > > ``` > > > > at > > > > > > > > ``` > > org.apache.lucene.codecs.DocValuesConsumer.mergeNumericField(DocValuesConsumer.java:129) > > > > > ``` > > > > at > > > > > > > > ``` > > org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:255) > > > > > ``` > > > > at > > > > > > > > ``` > > > > > > > > org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:133) > > > > at > > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4173) > > > > > ``` > > > > at > > > > > > > > ``` > > > > > > > > org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3768) > > > > at > > org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405) > > > > > ``` > > > > at > > > > > > > > ``` > > org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107) > > > > > ``` > > > > at > > > > > > > > ``` > > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482) > > > > > [2015-02-05 18:01:01,369][WARN][cluster.action.shard] > > > > [Kkallakku] > > > > [wa\_value\_index\_v1][0] sending failed shard for > > > > [wa\_value\_index\_v1][0], > > > > node[KpdH3su1QSyTx8lekwshAA], [P], s[STARTED], indexUUID > > > > [LgwNpUo7RRqjnIaVcA2rLA], reason [engine failure, message [merge > > > > exception][MergeException[java.lang.NullPointerException]; nested: > > > > NullPointerException; ]] > > > > [2015-02-05 18:01:01,369][WARN][cluster.action.shard] > > > > [Kkallakku] > > > > [wa\_value\_index\_v1][0] received shard failed for > > > > [wa\_value\_index\_v1][0], > > > > node[KpdH3su1QSyTx8lekwshAA], [P], s[STARTED], indexUUID > > > > [LgwNpUo7RRqjnIaVcA2rLA], reason [engine failure, message [merge > > > > exception][MergeException[java.lang.NullPointerException]; nested: > > > > NullPointerException; ]] > > > > > > > > org.elasticsearch.action.search.SearchPhaseExecutionException: Failed > > > > to > > > > execute phase [query\_fetch], all shards failed > > > > at > > org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:233) > > > > > at > > org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.start(TransportSearchTypeAction.java:156) > > > > > at > > org.elasticsearch.action.search.type.TransportSearchQueryAndFetchAction.doExecute(TransportSearchQueryAndFetchAction.java:55) > > > > > at > > org.elasticsearch.action.search.type.TransportSearchQueryAndFetchAction.doExecute(TransportSearchQueryAndFetchAction.java:45) > > > > > at > > org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:75) > > > > > at > > org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:107) > > > > > at > > org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:45) > > > > > Has any one seen these issues before? What are the causes? > > > > > > > > Many thanks, > > > > Cindy > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups > > > > "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, > > > > send > > > > an > > > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/637890eb-d50f-4932-a511-a3f7fb3f6aa9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/637890eb-d50f-4932-a511-a3f7fb3f6aa9%40googlegroups.com). > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google > > Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/31187132-7efd-4fa4-b31b-1e008f6933d6%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/31187132-7efd-4fa4-b31b-1e008f6933d6%40googlegroups.com). > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/82323ebe-327b-4a32-b811-e2bc1c192cdd%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/82323ebe-327b-4a32-b811-e2bc1c192cdd%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:34am UTC](https://discuss.elastic.co/t/mergeexception-java-lang-nullpointerexception-and-all-shards-failed-for-phase-query-fetch/22005/6 "2017-07-06T00:34:37Z")