[MergeException[java.lang.NullPointerException] and All shards failed for phase: [query_fetch]

ElasticSearch_Users_ · February 6, 2015, 12:01am

Hello,

My environment has 1 linux server installed elasticsearch 1.4.2 with
default settings from rpm package. I use TransportClient to send requests
to elasticsearch. I recently changed to use rpm package. It has been
working fine for a couple of day. But today I saw the following errors when
I indexed a data set I had successfully indexed before and tried to query a
simple word. In the meantime, the server takes much longer time to index
and delete indices.

[2015-02-05 18:01:01,105][WARN ][index.merge.scheduler ] [Kkallakku]
[wa_value_index_v1][0] failed to merge
java.lang.NullPointerException
at
org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer$NormMap.getOrd(Lucene49NormsConsumer.java:249)
at
org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer.addNumericField(Lucene49NormsConsumer.java:150)
at
org.apache.lucene.codecs.DocValuesConsumer.mergeNumericField(DocValuesConsumer.java:129)
at
org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:255)
at
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:133)
at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4173)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3768)
at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405)
at
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482)
[2015-02-05 18:01:01,106][WARN ][index.engine.internal ] [Kkallakku]
[wa_value_index_v1][0] failed engine [merge exception]
org.apache.lucene.index.MergePolicy$MergeException:
java.lang.NullPointerException
at
org.elasticsearch.index.merge.scheduler.ConcurrentMergeSchedulerProvider$CustomConcurrentMergeScheduler.handleMergeException(ConcurrentMergeSchedulerProvider.java:133)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:518)
Caused by: java.lang.NullPointerException
at
org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer$NormMap.getOrd(Lucene49NormsConsumer.java:249)
at
org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer.addNumericField(Lucene49NormsConsumer.java:150)
at
org.apache.lucene.codecs.DocValuesConsumer.mergeNumericField(DocValuesConsumer.java:129)
at
org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:255)
at
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:133)
at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4173)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3768)
at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405)
at
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482)
[2015-02-05 18:01:01,369][WARN ][cluster.action.shard ] [Kkallakku]
[wa_value_index_v1][0] sending failed shard for [wa_value_index_v1][0],
node[KpdH3su1QSyTx8lekwshAA], [P], s[STARTED], indexUUID
[LgwNpUo7RRqjnIaVcA2rLA], reason [engine failure, message [merge
exception][MergeException[java.lang.NullPointerException]; nested:
NullPointerException; ]]
[2015-02-05 18:01:01,369][WARN ][cluster.action.shard ] [Kkallakku]
[wa_value_index_v1][0] received shard failed for [wa_value_index_v1][0],
node[KpdH3su1QSyTx8lekwshAA], [P], s[STARTED], indexUUID
[LgwNpUo7RRqjnIaVcA2rLA], reason [engine failure, message [merge
exception][MergeException[java.lang.NullPointerException]; nested:
NullPointerException; ]]

org.elasticsearch.action.search.SearchPhaseExecutionException: Failed to
execute phase [query_fetch], all shards failed
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:233)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.start(TransportSearchTypeAction.java:156)
at
org.elasticsearch.action.search.type.TransportSearchQueryAndFetchAction.doExecute(TransportSearchQueryAndFetchAction.java:55)
at
org.elasticsearch.action.search.type.TransportSearchQueryAndFetchAction.doExecute(TransportSearchQueryAndFetchAction.java:45)
at
org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:75)
at
org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:107)
at
org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:45)

Has any one seen these issues before? What are the causes?

Many thanks,
Cindy

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/637890eb-d50f-4932-a511-a3f7fb3f6aa9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Robert_Muir_2 · February 6, 2015, 12:25am

Are you using an IBM JDK? Don't do that

On Thu, Feb 5, 2015 at 7:01 PM, 'Cindy' via elasticsearch
elasticsearch@googlegroups.com wrote:

Hello,

My environment has 1 linux server installed elasticsearch 1.4.2 with default
settings from rpm package. I use TransportClient to send requests to
elasticsearch. I recently changed to use rpm package. It has been working
fine for a couple of day. But today I saw the following errors when I
indexed a data set I had successfully indexed before and tried to query a
simple word. In the meantime, the server takes much longer time to index and
delete indices.

[2015-02-05 18:01:01,105][WARN ][index.merge.scheduler ] [Kkallakku]
[wa_value_index_v1][0] failed to merge
java.lang.NullPointerException
at
org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer$NormMap.getOrd(Lucene49NormsConsumer.java:249)
at
org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer.addNumericField(Lucene49NormsConsumer.java:150)
at
org.apache.lucene.codecs.DocValuesConsumer.mergeNumericField(DocValuesConsumer.java:129)
at
org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:255)
at
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:133)
at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4173)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3768)
at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405)
at
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482)
[2015-02-05 18:01:01,106][WARN ][index.engine.internal ] [Kkallakku]
[wa_value_index_v1][0] failed engine [merge exception]
org.apache.lucene.index.MergePolicy$MergeException:
java.lang.NullPointerException
at
org.elasticsearch.index.merge.scheduler.ConcurrentMergeSchedulerProvider$CustomConcurrentMergeScheduler.handleMergeException(ConcurrentMergeSchedulerProvider.java:133)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:518)
Caused by: java.lang.NullPointerException
at
org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer$NormMap.getOrd(Lucene49NormsConsumer.java:249)
at
org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer.addNumericField(Lucene49NormsConsumer.java:150)
at
org.apache.lucene.codecs.DocValuesConsumer.mergeNumericField(DocValuesConsumer.java:129)
at
org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:255)
at
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:133)
at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4173)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3768)
at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405)
at
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482)
[2015-02-05 18:01:01,369][WARN ][cluster.action.shard ] [Kkallakku]
[wa_value_index_v1][0] sending failed shard for [wa_value_index_v1][0],
node[KpdH3su1QSyTx8lekwshAA], [P], s[STARTED], indexUUID
[LgwNpUo7RRqjnIaVcA2rLA], reason [engine failure, message [merge
exception][MergeException[java.lang.NullPointerException]; nested:
NullPointerException; ]]
[2015-02-05 18:01:01,369][WARN ][cluster.action.shard ] [Kkallakku]
[wa_value_index_v1][0] received shard failed for [wa_value_index_v1][0],
node[KpdH3su1QSyTx8lekwshAA], [P], s[STARTED], indexUUID
[LgwNpUo7RRqjnIaVcA2rLA], reason [engine failure, message [merge
exception][MergeException[java.lang.NullPointerException]; nested:
NullPointerException; ]]

org.elasticsearch.action.search.SearchPhaseExecutionException: Failed to
execute phase [query_fetch], all shards failed
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:233)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.start(TransportSearchTypeAction.java:156)
at
org.elasticsearch.action.search.type.TransportSearchQueryAndFetchAction.doExecute(TransportSearchQueryAndFetchAction.java:55)
at
org.elasticsearch.action.search.type.TransportSearchQueryAndFetchAction.doExecute(TransportSearchQueryAndFetchAction.java:45)
at
org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:75)
at
org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:107)
at
org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:45)

Has any one seen these issues before? What are the causes?

Many thanks,
Cindy

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/637890eb-d50f-4932-a511-a3f7fb3f6aa9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAMUKNZVROAeQjM1UKCgdxXs52S81Yaad74%3D4kDdpeOXarNf1aA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

ElasticSearch_Users_ · February 6, 2015, 12:39am

Yes, I am using IBM JDK. How do you detect it that? Does it show from the
log?

I know there is a Lucene IBM JVM bug. Is this related? What are the cause
of the above exceptions?

Thank you,

Cindy

On Thursday, 5 February 2015 19:25:49 UTC-5, Robert Muir wrote:

Are you using an IBM JDK? Don't do that

On Thu, Feb 5, 2015 at 7:01 PM, 'Cindy' via elasticsearch
<elasti...@googlegroups.com <javascript:>> wrote:

Hello,

My environment has 1 linux server installed elasticsearch 1.4.2 with
default
settings from rpm package. I use TransportClient to send requests to
elasticsearch. I recently changed to use rpm package. It has been
working
fine for a couple of day. But today I saw the following errors when I
indexed a data set I had successfully indexed before and tried to query
a
simple word. In the meantime, the server takes much longer time to index
and
delete indices.

[2015-02-05 18:01:01,105][WARN ][index.merge.scheduler ] [Kkallakku]
[wa_value_index_v1][0] failed to merge
java.lang.NullPointerException
at

org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer$NormMap.getOrd(Lucene49NormsConsumer.java:249)
    at 
org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer.addNumericField(Lucene49NormsConsumer.java:150)
    at 
org.apache.lucene.codecs.DocValuesConsumer.mergeNumericField(DocValuesConsumer.java:129)
    at 
org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:255)
at
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:133)
at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4173)
at
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3768)
at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405)
    at 
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
    at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482)

[2015-02-05 18:01:01,106][WARN ][index.engine.internal ] [Kkallakku]
[wa_value_index_v1][0] failed engine [merge exception]
org.apache.lucene.index.MergePolicy$MergeException:
java.lang.NullPointerException
at

org.elasticsearch.index.merge.scheduler.ConcurrentMergeSchedulerProvider$CustomConcurrentMergeScheduler.handleMergeException(ConcurrentMergeSchedulerProvider.java:133)
    at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:518)

Caused by: java.lang.NullPointerException
at

org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer$NormMap.getOrd(Lucene49NormsConsumer.java:249)
    at 
org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer.addNumericField(Lucene49NormsConsumer.java:150)
    at 
org.apache.lucene.codecs.DocValuesConsumer.mergeNumericField(DocValuesConsumer.java:129)
    at 
org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:255)
at
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:133)
at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4173)
at
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3768)
at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405)
    at 
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
    at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482)

[2015-02-05 18:01:01,369][WARN ][cluster.action.shard ] [Kkallakku]
[wa_value_index_v1][0] sending failed shard for [wa_value_index_v1][0],
node[KpdH3su1QSyTx8lekwshAA], [P], s[STARTED], indexUUID
[LgwNpUo7RRqjnIaVcA2rLA], reason [engine failure, message [merge
exception][MergeException[java.lang.NullPointerException]; nested:
NullPointerException; ]]
[2015-02-05 18:01:01,369][WARN ][cluster.action.shard ] [Kkallakku]
[wa_value_index_v1][0] received shard failed for [wa_value_index_v1][0],
node[KpdH3su1QSyTx8lekwshAA], [P], s[STARTED], indexUUID
[LgwNpUo7RRqjnIaVcA2rLA], reason [engine failure, message [merge
exception][MergeException[java.lang.NullPointerException]; nested:
NullPointerException; ]]

org.elasticsearch.action.search.SearchPhaseExecutionException: Failed to
execute phase [query_fetch], all shards failed
at

org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:233)

at

org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.start(TransportSearchTypeAction.java:156)

at

org.elasticsearch.action.search.type.TransportSearchQueryAndFetchAction.doExecute(TransportSearchQueryAndFetchAction.java:55)

at

org.elasticsearch.action.search.type.TransportSearchQueryAndFetchAction.doExecute(TransportSearchQueryAndFetchAction.java:45)

at

org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:75)

at

org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:107)

at

org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:45)

Has any one seen these issues before? What are the causes?

Many thanks,
Cindy

--
You received this message because you are subscribed to the Google
Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit

https://groups.google.com/d/msgid/elasticsearch/637890eb-d50f-4932-a511-a3f7fb3f6aa9%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/31187132-7efd-4fa4-b31b-1e008f6933d6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Robert_Muir_2 · February 6, 2015, 12:43am

I know this because I have seen this same IBM JDK bug in our tests many times.

I also wrote the code in question. NPE is not possible.
https://github.com/apache/lucene-solr/blob/lucene_solr_4_10_2/lucene/core/src/java/org/apache/lucene/codecs/lucene49/Lucene49NormsConsumer.java#L213

This is why we say, don't use IBM JDK with lucene.

On Thu, Feb 5, 2015 at 7:39 PM, 'Cindy' via elasticsearch
elasticsearch@googlegroups.com wrote:

Yes, I am using IBM JDK. How do you detect it that? Does it show from the
log?

I know there is a Lucene IBM JVM bug. Is this related? What are the cause of
the above exceptions?

Thank you,

Cindy

On Thursday, 5 February 2015 19:25:49 UTC-5, Robert Muir wrote:

Are you using an IBM JDK? Don't do that

On Thu, Feb 5, 2015 at 7:01 PM, 'Cindy' via elasticsearch
elasti...@googlegroups.com wrote:

Hello,

My environment has 1 linux server installed elasticsearch 1.4.2 with
default
settings from rpm package. I use TransportClient to send requests to
elasticsearch. I recently changed to use rpm package. It has been
working
fine for a couple of day. But today I saw the following errors when I
indexed a data set I had successfully indexed before and tried to query
a
simple word. In the meantime, the server takes much longer time to index
and
delete indices.

[2015-02-05 18:01:01,105][WARN ][index.merge.scheduler ] [Kkallakku]
[wa_value_index_v1][0] failed to merge
java.lang.NullPointerException
at

org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer$NormMap.getOrd(Lucene49NormsConsumer.java:249)
at

org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer.addNumericField(Lucene49NormsConsumer.java:150)
at

org.apache.lucene.codecs.DocValuesConsumer.mergeNumericField(DocValuesConsumer.java:129)
at
org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:255)
at
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:133)
at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4173)
at
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3768)
at

org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405)
at

org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
at

org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482)
[2015-02-05 18:01:01,106][WARN ][index.engine.internal ] [Kkallakku]
[wa_value_index_v1][0] failed engine [merge exception]
org.apache.lucene.index.MergePolicy$MergeException:
java.lang.NullPointerException
at

org.elasticsearch.index.merge.scheduler.ConcurrentMergeSchedulerProvider$CustomConcurrentMergeScheduler.handleMergeException(ConcurrentMergeSchedulerProvider.java:133)
at

org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:518)
Caused by: java.lang.NullPointerException
at

org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer$NormMap.getOrd(Lucene49NormsConsumer.java:249)
at

org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer.addNumericField(Lucene49NormsConsumer.java:150)
at

org.apache.lucene.codecs.DocValuesConsumer.mergeNumericField(DocValuesConsumer.java:129)
at
org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:255)
at
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:133)
at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4173)
at
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3768)
at

org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405)
at

org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
at

org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482)
[2015-02-05 18:01:01,369][WARN ][cluster.action.shard ] [Kkallakku]
[wa_value_index_v1][0] sending failed shard for [wa_value_index_v1][0],
node[KpdH3su1QSyTx8lekwshAA], [P], s[STARTED], indexUUID
[LgwNpUo7RRqjnIaVcA2rLA], reason [engine failure, message [merge
exception][MergeException[java.lang.NullPointerException]; nested:
NullPointerException; ]]
[2015-02-05 18:01:01,369][WARN ][cluster.action.shard ] [Kkallakku]
[wa_value_index_v1][0] received shard failed for [wa_value_index_v1][0],
node[KpdH3su1QSyTx8lekwshAA], [P], s[STARTED], indexUUID
[LgwNpUo7RRqjnIaVcA2rLA], reason [engine failure, message [merge
exception][MergeException[java.lang.NullPointerException]; nested:
NullPointerException; ]]

org.elasticsearch.action.search.SearchPhaseExecutionException: Failed to
execute phase [query_fetch], all shards failed
at

org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:233)
at

org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.start(TransportSearchTypeAction.java:156)
at

org.elasticsearch.action.search.type.TransportSearchQueryAndFetchAction.doExecute(TransportSearchQueryAndFetchAction.java:55)
at

org.elasticsearch.action.search.type.TransportSearchQueryAndFetchAction.doExecute(TransportSearchQueryAndFetchAction.java:45)
at

org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:75)
at

org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:107)
at

org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:45)

Has any one seen these issues before? What are the causes?

Many thanks,
Cindy

--
You received this message because you are subscribed to the Google
Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit

https://groups.google.com/d/msgid/elasticsearch/637890eb-d50f-4932-a511-a3f7fb3f6aa9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/31187132-7efd-4fa4-b31b-1e008f6933d6%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAMUKNZX%2BZLRNW8KYwq0mERj%3DfFAv34f6PeOazd36AaOQ-pSmaA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

ElasticSearch_Users_ · February 6, 2015, 12:53am

It's really appreciated that you quickly pointed out the cause!!!

Actually I had posted a question in this forum trying to find out more
details about the IBM JVM bug and why I can't use IBM JVM. Now I knew it

On Thursday, 5 February 2015 19:44:26 UTC-5, Robert Muir wrote:

I know this because I have seen this same IBM JDK bug in our tests many
times.

I also wrote the code in question. NPE is not possible.

https://github.com/apache/lucene-solr/blob/lucene_solr_4_10_2/lucene/core/src/java/org/apache/lucene/codecs/lucene49/Lucene49NormsConsumer.java#L213

This is why we say, don't use IBM JDK with lucene.

On Thu, Feb 5, 2015 at 7:39 PM, 'Cindy' via elasticsearch
<elasti...@googlegroups.com <javascript:>> wrote:

Yes, I am using IBM JDK. How do you detect it that? Does it show from
the
log?

I know there is a Lucene IBM JVM bug. Is this related? What are the
cause of
the above exceptions?

Thank you,

Cindy

On Thursday, 5 February 2015 19:25:49 UTC-5, Robert Muir wrote:

Are you using an IBM JDK? Don't do that

On Thu, Feb 5, 2015 at 7:01 PM, 'Cindy' via elasticsearch
elasti...@googlegroups.com wrote:

Hello,

My environment has 1 linux server installed elasticsearch 1.4.2 with
default
settings from rpm package. I use TransportClient to send requests to
elasticsearch. I recently changed to use rpm package. It has been
working
fine for a couple of day. But today I saw the following errors when I
indexed a data set I had successfully indexed before and tried to
query
a
simple word. In the meantime, the server takes much longer time to
index
and
delete indices.

[2015-02-05 18:01:01,105][WARN ][index.merge.scheduler ]
[Kkallakku]
[wa_value_index_v1][0] failed to merge
java.lang.NullPointerException
at

org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer$NormMap.getOrd(Lucene49NormsConsumer.java:249)
    at 
org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer.addNumericField(Lucene49NormsConsumer.java:150)
    at 
org.apache.lucene.codecs.DocValuesConsumer.mergeNumericField(DocValuesConsumer.java:129)
    at 
org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:255)
    at 
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:133)
at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4173)
    at 
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3768)
at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405)
    at 
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
    at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482)

[2015-02-05 18:01:01,106][WARN ][index.engine.internal ]
[Kkallakku]
[wa_value_index_v1][0] failed engine [merge exception]
org.apache.lucene.index.MergePolicy$MergeException:
java.lang.NullPointerException
at

org.elasticsearch.index.merge.scheduler.ConcurrentMergeSchedulerProvider$CustomConcurrentMergeScheduler.handleMergeException(ConcurrentMergeSchedulerProvider.java:133)
    at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:518)

Caused by: java.lang.NullPointerException
at

org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer$NormMap.getOrd(Lucene49NormsConsumer.java:249)
    at 
org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer.addNumericField(Lucene49NormsConsumer.java:150)
    at 
org.apache.lucene.codecs.DocValuesConsumer.mergeNumericField(DocValuesConsumer.java:129)
    at 
org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:255)
    at 
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:133)
at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4173)
    at 
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3768)
at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405)
    at 
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
    at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482)

[2015-02-05 18:01:01,369][WARN ][cluster.action.shard ]
[Kkallakku]
[wa_value_index_v1][0] sending failed shard for
[wa_value_index_v1][0],
node[KpdH3su1QSyTx8lekwshAA], [P], s[STARTED], indexUUID
[LgwNpUo7RRqjnIaVcA2rLA], reason [engine failure, message [merge
exception][MergeException[java.lang.NullPointerException]; nested:
NullPointerException; ]]
[2015-02-05 18:01:01,369][WARN ][cluster.action.shard ]
[Kkallakku]
[wa_value_index_v1][0] received shard failed for
[wa_value_index_v1][0],
node[KpdH3su1QSyTx8lekwshAA], [P], s[STARTED], indexUUID
[LgwNpUo7RRqjnIaVcA2rLA], reason [engine failure, message [merge
exception][MergeException[java.lang.NullPointerException]; nested:
NullPointerException; ]]

org.elasticsearch.action.search.SearchPhaseExecutionException: Failed
to
execute phase [query_fetch], all shards failed
at

org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:233)

at

org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.start(TransportSearchTypeAction.java:156)

at

org.elasticsearch.action.search.type.TransportSearchQueryAndFetchAction.doExecute(TransportSearchQueryAndFetchAction.java:55)

at

org.elasticsearch.action.search.type.TransportSearchQueryAndFetchAction.doExecute(TransportSearchQueryAndFetchAction.java:45)

at

org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:75)

at

org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:107)

at

org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:45)

Has any one seen these issues before? What are the causes?

Many thanks,
Cindy

--
You received this message because you are subscribed to the Google
Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send
an
email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit

https://groups.google.com/d/msgid/elasticsearch/637890eb-d50f-4932-a511-a3f7fb3f6aa9%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit

https://groups.google.com/d/msgid/elasticsearch/31187132-7efd-4fa4-b31b-1e008f6933d6%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/82323ebe-327b-4a32-b811-e2bc1c192cdd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.