Hi
We have an elasticsearch cluster, which produces a very very high cpu usage
, all the nodes are fully utilize all the time.
When the thread pools (search thread pool specifically) get full, the
cluster stops responding until we hard restart all the nodes.
We'll appreciate any help to avoid thses situations. Details below:
Thanks,
Nitzan.
elasticsearch version - 1.3.4
java version -
Oracle java version "1.7.0_67"
Java(TM) SE Runtime Environment (build 1.7.0_67-b01)
Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)
5 nodes
2 Indices
shards - 61
java heap size - 30G
documents number - 485042729
ec2 machines
instance type - c3.8xlarge
vCPU - 32
RAM - 60GB
SSD 1000 iops for data store
Operating system -
Ubuntu precise 12.04
hot threads:
curl -XGET http://localhost:9200/_nodes/hot_threads
::: [ES1][SmFW31WOTH-Nr-0VFpXTag][ES1][inet[
/10.0.1.10:9300]]{max_local_storage_nodes=1}
47.4% (236.9ms out of 500ms) cpu usage by thread
'elasticsearch[ES1][search][T#93]'
6/10 snapshots sharing following 14 elements
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.
java:193)
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:
163)
org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)
org.elasticsearch.search.internal.ContextIndexSearcher.search(
ContextIndexSearcher.java:175)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:116
)
org.elasticsearch.search.SearchService.executeQueryPhase(
SearchService.java:262)
org.elasticsearch.search.action.SearchServiceTransportAction$5.call(
SearchServiceTransportAction.java:206)
org.elasticsearch.search.action.SearchServiceTransportAction$5.call(
SearchServiceTransportAction.java:203)
org.elasticsearch.search.action.SearchServiceTransportAction$23.run(
SearchServiceTransportAction.java:517)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:615)
java.lang.Thread.run(Thread.java:745)
4/10 snapshots sharing following 15 elements
org.apache.lucene.search.ConstantScoreQuery$ConstantScorer.nextDoc(
ConstantScoreQuery.java:257)
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.
java:192)
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:
163)
org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)
org.elasticsearch.search.internal.ContextIndexSearcher.search(
ContextIndexSearcher.java:175)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:116
)
org.elasticsearch.search.SearchService.executeQueryPhase(
SearchService.java:262)
org.elasticsearch.search.action.SearchServiceTransportAction$5.call(
SearchServiceTransportAction.java:206)
org.elasticsearch.search.action.SearchServiceTransportAction$5.call(
SearchServiceTransportAction.java:203)
org.elasticsearch.search.action.SearchServiceTransportAction$23.run(
SearchServiceTransportAction.java:517)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:615)
java.lang.Thread.run(Thread.java:745)
42.8% (213.8ms out of 500ms) cpu usage by thread
'elasticsearch[ES1][search][T#85]'
6/10 snapshots sharing following 15 elements
org.apache.lucene.search.ConstantScoreQuery$ConstantScorer.nextDoc(
ConstantScoreQuery.java:257)
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.
java:192)
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:
163)
org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)
org.elasticsearch.search.internal.ContextIndexSearcher.search(
ContextIndexSearcher.java:175)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:116
)
org.elasticsearch.search.SearchService.executeQueryPhase(
SearchService.java:262)
org.elasticsearch.search.action.SearchServiceTransportAction$5.call(
SearchServiceTransportAction.java:206)
org.elasticsearch.search.action.SearchServiceTransportAction$5.call(
SearchServiceTransportAction.java:203)
org.elasticsearch.search.action.SearchServiceTransportAction$23.run(
SearchServiceTransportAction.java:517)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:615)
java.lang.Thread.run(Thread.java:745)
4/10 snapshots sharing following 15 elements
org.elasticsearch.common.lucene.search.FilteredCollector.collect(
FilteredCollector.java:60)
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.
java:193)
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:
163)
org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)
org.elasticsearch.search.internal.ContextIndexSearcher.search(
ContextIndexSearcher.java:175)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:116
)
org.elasticsearch.search.SearchService.executeQueryPhase(
SearchService.java:262)
org.elasticsearch.search.action.SearchServiceTransportAction$5.call(
SearchServiceTransportAction.java:206)
org.elasticsearch.search.action.SearchServiceTransportAction$5.call(
SearchServiceTransportAction.java:203)
org.elasticsearch.search.action.SearchServiceTransportAction$23.run(
SearchServiceTransportAction.java:517)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:615)
java.lang.Thread.run(Thread.java:745)
42.7% (213.3ms out of 500ms) cpu usage by thread
'elasticsearch[ES1][search][T#26]'
5/10 snapshots sharing following 16 elements
org.apache.lucene.search.FilteredDocIdSetIterator.nextDoc(
FilteredDocIdSetIterator.java:59)
org.apache.lucene.search.ConstantScoreQuery$ConstantScorer.nextDoc(
ConstantScoreQuery.java:257)
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.
java:192)
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:
163)
org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)
org.elasticsearch.search.internal.ContextIndexSearcher.search(
ContextIndexSearcher.java:175)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:116
)
org.elasticsearch.search.SearchService.executeQueryPhase(
SearchService.java:262)
org.elasticsearch.search.action.SearchServiceTransportAction$5.call(
SearchServiceTransportAction.java:206)
org.elasticsearch.search.action.SearchServiceTransportAction$5.call(
SearchServiceTransportAction.java:203)
org.elasticsearch.search.action.SearchServiceTransportAction$23.run(
SearchServiceTransportAction.java:517)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:615)
java.lang.Thread.run(Thread.java:745)
5/10 snapshots sharing following 15 elements
org.apache.lucene.search.ConstantScoreQuery$ConstantScorer.nextDoc(
ConstantScoreQuery.java:257)
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.
java:192)
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:
163)
org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)
org.elasticsearch.search.internal.ContextIndexSearcher.search(
ContextIndexSearcher.java:175)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:116
)
org.elasticsearch.search.SearchService.executeQueryPhase(
SearchService.java:262)
org.elasticsearch.search.action.SearchServiceTransportAction$5.call(
SearchServiceTransportAction.java:206)
org.elasticsearch.search.action.SearchServiceTransportAction$5.call(
SearchServiceTransportAction.java:203)
org.elasticsearch.search.action.SearchServiceTransportAction$23.run(
SearchServiceTransportAction.java:517)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:615)
java.lang.Thread.run(Thread.java:745)
::: [ES3][Gb8cjXtWRUShqdnVUhPEqw][ES3][inet[
/10.0.1.11:9300]]{max_local_storage_nodes=1}
42.2% (211.1ms out of 500ms) cpu usage by thread
'elasticsearch[ES3][search][T#15]'
7/10 snapshots sharing following 16 elements
org.apache.lucene.search.FilteredDocIdSetIterator.nextDoc(
FilteredDocIdSetIterator.java:60)
org.apache.lucene.search.ConstantScoreQuery$ConstantScorer.nextDoc(
ConstantScoreQuery.java:257)
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.
java:192)
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:
163)
org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)
org.elasticsearch.search.internal.ContextIndexSearcher.search(
ContextIndexSearcher.java:175)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:116
)
org.elasticsearch.search.SearchService.executeQueryPhase(
SearchService.java:262)
org.elasticsearch.search.action.
SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(
SearchServiceTransportAction.java:688)
org.elasticsearch.search.action.
SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(
SearchServiceTransportAction.java:677)
org.elasticsearch.transport.netty.
MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:275)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:615)
java.lang.Thread.run(Thread.java:745)
3/10 snapshots sharing following 15 elements
org.elasticsearch.common.lucene.search.FilteredCollector.collect(
FilteredCollector.java:60)
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.
java:193)
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:
163)
org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)
org.elasticsearch.search.internal.ContextIndexSearcher.search(
ContextIndexSearcher.java:175)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:116
)
org.elasticsearch.search.SearchService.executeQueryPhase(
SearchService.java:262)
org.elasticsearch.search.action.
SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(
SearchServiceTransportAction.java:688)
org.elasticsearch.search.action.
SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(
SearchServiceTransportAction.java:677)
org.elasticsearch.transport.netty.
MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:275)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:615)
java.lang.Thread.run(Thread.java:745)
42.2% (210.8ms out of 500ms) cpu usage by thread
'elasticsearch[ES3][search][T#67]'
3/10 snapshots sharing following 19 elements
org.apache.lucene.search.FilteredDocIdSetIterator.nextDoc(
FilteredDocIdSetIterator.java:59)
org.apache.lucene.search.ConstantScoreQuery$ConstantScorer.nextDoc(
ConstantScoreQuery.java:257)
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.
java:192)
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:
163)
org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)
org.elasticsearch.search.internal.ContextIndexSearcher.search(
ContextIndexSearcher.java:170)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:581)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:533)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:510)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:345)
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:149
)
org.elasticsearch.search.SearchService.executeQueryPhase(
SearchService.java:262)
org.elasticsearch.search.action.
SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(
SearchServiceTransportAction.java:688)
org.elasticsearch.search.action.
SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(
SearchServiceTransportAction.java:677)
org.elasticsearch.transport.netty.
MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:275)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:615)
java.lang.Thread.run(Thread.java:745)
5/10 snapshots sharing following 18 elements
org.elasticsearch.common.lucene.search.FilteredCollector.collect(
FilteredCollector.java:60)
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.
java:193)
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:
163)
org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)
org.elasticsearch.search.internal.ContextIndexSearcher.search(
ContextIndexSearcher.java:170)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:581)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:533)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:510)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:345)
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:149
)
org.elasticsearch.search.SearchService.executeQueryPhase(
SearchService.java:262)
org.elasticsearch.search.action.
SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(
SearchServiceTransportAction.java:688)
org.elasticsearch.search.action.
SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(
SearchServiceTransportAction.java:677)
org.elasticsearch.transport.netty.
MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:275)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:615)
java.lang.Thread.run(Thread.java:745)
2/10 snapshots sharing following 19 elements
org.apache.lucene.search.FilteredDocIdSetIterator.nextDoc(
FilteredDocIdSetIterator.java:60)
org.apache.lucene.search.ConstantScoreQuery$ConstantScorer.nextDoc(
ConstantScoreQuery.java:257)
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.
java:192)
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:
163)
org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)
org.elasticsearch.search.internal.ContextIndexSearcher.search(
ContextIndexSearcher.java:170)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:581)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:533)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:510)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:345)
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:149
)
org.elasticsearch.search.SearchService.executeQueryPhase(
SearchService.java:262)
org.elasticsearch.search.action.
SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(
SearchServiceTransportAction.java:688)
org.elasticsearch.search.action.
SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(
SearchServiceTransportAction.java:677)
org.elasticsearch.transport.netty.
MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:275)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:615)
java.lang.Thread.run(Thread.java:745)
41.9% (209.6ms out of 500ms) cpu usage by thread
'elasticsearch[ES3][search][T#66]'
8/10 snapshots sharing following 18 elements
org.elasticsearch.common.lucene.search.FilteredCollector.collect(
FilteredCollector.java:60)
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.
java:193)
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:
163)
org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)
org.elasticsearch.search.internal.ContextIndexSearcher.search(
ContextIndexSearcher.java:175)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:581)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:533)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:510)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:345)
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:149
)
org.elasticsearch.search.SearchService.executeQueryPhase(
SearchService.java:262)
org.elasticsearch.search.action.
SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(
SearchServiceTransportAction.java:688)
org.elasticsearch.search.action.
SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(
SearchServiceTransportAction.java:677)
org.elasticsearch.transport.netty.
MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:275)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:615)
java.lang.Thread.run(Thread.java:745)
2/10 snapshots sharing following 19 elements
org.apache.lucene.search.FilteredDocIdSetIterator.nextDoc(
FilteredDocIdSetIterator.java:60)
org.apache.lucene.search.ConstantScoreQuery$ConstantScorer.nextDoc(
ConstantScoreQuery.java:257)
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.
java:192)
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:
163)
org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)
org.elasticsearch.search.internal.ContextIndexSearcher.search(
ContextIndexSearcher.java:175)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:581)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:533)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:510)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:345)
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:149
)
org.elasticsearch.search.SearchService.executeQueryPhase(
SearchService.java:262)
org.elasticsearch.search.action.
SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(
SearchServiceTransportAction.java:688)
org.elasticsearch.search.action.
SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(
SearchServiceTransportAction.java:677)
org.elasticsearch.transport.netty.
MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:275)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:615)
java.lang.Thread.run(Thread.java:745)
::: [ES5][q_VWePETRIOLS6IxvK3CxQ][ES5][inet[
/10.0.1.12:9300]]{max_local_storage_nodes=1}
38.3% (191.4ms out of 500ms) cpu usage by thread
'elasticsearch[ES5][search][T#40]'
7/10 snapshots sharing following 15 elements
org.elasticsearch.common.lucene.search.FilteredCollector.collect(
FilteredCollector.java:60)
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.
java:193)
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:
163)
org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)
org.elasticsearch.search.internal.ContextIndexSearcher.search(
ContextIndexSearcher.java:175)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:116
)
org.elasticsearch.search.SearchService.executeQueryPhase(
SearchService.java:262)
org.elasticsearch.search.action.
SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(
SearchServiceTransportAction.java:688)
org.elasticsearch.search.action.
SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(
SearchServiceTransportAction.java:677)
org.elasticsearch.transport.netty.
MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:275)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:615)
java.lang.Thread.run(Thread.java:745)
3/10 snapshots sharing following 3 elements
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:615)
java.lang.Thread.run(Thread.java:745)
37.5% (187.2ms out of 500ms) cpu usage by thread
'elasticsearch[ES5][search][T#15]'
2/10 snapshots sharing following 18 elements
org.elasticsearch.common.lucene.search.FilteredCollector.collect(
FilteredCollector.java:60)
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.
java:193)
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:
163)
org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)
org.elasticsearch.search.internal.ContextIndexSearcher.search(
ContextIndexSearcher.java:175)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:581)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:533)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:510)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:345)
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:149
)
org.elasticsearch.search.SearchService.executeQueryPhase(
SearchService.java:262)
org.elasticsearch.search.action.
SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(
SearchServiceTransportAction.java:688)
org.elasticsearch.search.action.
SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(
SearchServiceTransportAction.java:677)
org.elasticsearch.transport.netty.
MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:275)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:615)
java.lang.Thread.run(Thread.java:745)
5/10 snapshots sharing following 19 elements
org.apache.lucene.search.FilteredDocIdSetIterator.nextDoc(
FilteredDocIdSetIterator.java:60)
org.apache.lucene.search.ConstantScoreQuery$ConstantScorer.nextDoc(
ConstantScoreQuery.java:257)
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.
java:192)
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:
163)
org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)
org.elasticsearch.search.internal.ContextIndexSearcher.search(
ContextIndexSearcher.java:175)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:581)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:533)
org.apache.lucene.se
...
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8f7f9112-2011-49ba-b382-1886c8216602%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
