My Elasticsearch is running at very high CPU (constantly 99%) - Need help understanding hot_threads

Melanie_Zamora · August 4, 2014, 4:46pm

Hello,

My ES node is running very high and this is the thread from running
hot_threads. Can someone help me understand why CPU is so high?

This is a 1 node cluster running a linux machine with 16 cores and 72GB of
RAM.

::: [c9q9o027][VTde-tWsTcm9oe7vBZuadA][inet[/10.1.6.21:9300]]

48.3% (241.3ms out of 500ms) cpu usage by thread 'elasticsearch[c9q9o027][search][T#3]'
10/10 snapshots sharing following 17 elements
org.apache.lucene.search.ConstantScoreQuery$ConstantScorer.score(ConstantScoreQuery.java:245)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:627)
org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:167)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:491)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:281)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:269)
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:119)
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:233)
org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)
java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
java.lang.Thread.run(Unknown Source)

48.3% (241.3ms out of 500ms) cpu usage by thread 'elasticsearch[c9q9o027][search][T#8]'
2/10 snapshots sharing following 23 elements
org.apache.lucene.search.ConstantScoreQuery$ConstantScorer.docID(ConstantScoreQuery.java:190)
org.apache.lucene.search.DisjunctionScorer.heapAdjust(DisjunctionScorer.java:58)
org.apache.lucene.search.DisjunctionSumScorer.nextDoc(DisjunctionSumScorer.java:54)
org.elasticsearch.common.lucene.docset.DocIdSets.toFixedBitSet(DocIdSets.java:143)
org.elasticsearch.common.lucene.docset.DocIdSets.toSafeBits(DocIdSets.java:134)
org.elasticsearch.common.lucene.search.FilteredCollector.setNextReader(FilteredCollector.java:69)
org.elasticsearch.common.lucene.MultiCollector.setNextReader(MultiCollector.java:68)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:618)
org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:167)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:491)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:281)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:269)
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:119)
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:233)
org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)
java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
java.lang.Thread.run(Unknown Source)
8/10 snapshots sharing following 23 elements
org.apache.lucene.search.ConstantScoreQuery$ConstantScorer.docID(ConstantScoreQuery.java:190)
org.apache.lucene.search.DisjunctionSumScorer.nextDoc(DisjunctionSumScorer.java:61)
org.apache.lucene.search.DisjunctionSumScorer.nextDoc(DisjunctionSumScorer.java:53)
org.elasticsearch.common.lucene.docset.DocIdSets.toFixedBitSet(DocIdSets.java:143)
org.elasticsearch.common.lucene.docset.DocIdSets.toSafeBits(DocIdSets.java:134)
org.elasticsearch.common.lucene.search.FilteredCollector.setNextReader(FilteredCollector.java:69)
org.elasticsearch.common.lucene.MultiCollector.setNextReader(MultiCollector.java:68)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:618)
org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:167)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:491)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:281)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:269)
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:119)
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:233)
org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)
java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
java.lang.Thread.run(Unknown Source)

39.8% (199.2ms out of 500ms) cpu usage by thread 'elasticsearch[c9q9o027][search][T#15]'
10/10 snapshots sharing following 23 elements
org.elasticsearch.common.lucene.search.XBooleanFilter.getDocIdSet(XBooleanFilter.java:224)
org.elasticsearch.common.lucene.search.ApplyAcceptedDocsFilter.getDocIdSet(ApplyAcceptedDocsFilter.java:45)
org.apache.lucene.search.FilteredQuery$1.scorer(FilteredQuery.java:128)
org.apache.lucene.search.QueryWrapperFilter$1.iterator(QueryWrapperFilter.java:59)
org.elasticsearch.common.lucene.docset.DocIdSets.toSafeBits(DocIdSets.java:130)
org.elasticsearch.common.lucene.search.FilteredCollector.setNextReader(FilteredCollector.java:69)
org.elasticsearch.common.lucene.MultiCollector.setNextReader(MultiCollector.java:68)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:618)
org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:167)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:491)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:281)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:269)
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:119)
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:233)
org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)
java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
java.lang.Thread.run(Unknown Source)

::: [DeWolff, Jean][Z2ezcYq2QDiEwHjnbsLiYQ][inet[/10.1.6.21:9301]]{client=true, data=false}

0.9% (4.6ms out of 500ms) cpu usage by thread '<redis'
 10/10 snapshots sharing following 35 elements
   java.lang.Object.wait(Native Method)
   org.jruby.RubyThread.sleep(RubyThread.java:864)
   org.jruby.RubyKernel.sleep(RubyKernel.java:801)
   org.jruby.RubyKernel$INVOKER$s$0$1$sleep.call(RubyKernel$INVOKER$s$0$1$sleep.gen)
   org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:646)
   org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:205)
   org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:168)
   org.jruby.ast.FCallOneArgNode.interpret(FCallOneArgNode.java:36)
   org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
   org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
   org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
   org.jruby.ast.RescueBodyNode.interpret(RescueBodyNode.java:108)
   org.jruby.ast.RescueNode.handleException(RescueNode.java:177)
   org.jruby.ast.RescueNode.interpret(RescueNode.java:123)
   org.jruby.ast.BeginNode.interpret(BeginNode.java:83)
   org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
   org.jruby.ast.WhileNode.interpret(WhileNode.java:131)
   org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
   org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
   org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:75)
   org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:225)
   org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:204)
   org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:346)
   org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:204)
   org.jruby.ast.FCallSpecialArgNode.interpret(FCallSpecialArgNode.java:41)
   org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
   org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112)
   org.jruby.runtime.Interpreted19Block.evalBlockBody(Interpreted19Block.java:209)
   org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:197)
   org.jruby.runtime.Interpreted19Block.call(Interpreted19Block.java:128)
   org.jruby.runtime.Block.call(Block.java:89)
   org.jruby.RubyProc.call(RubyProc.java:261)
   org.jruby.RubyProc.call(RubyProc.java:213)
   org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:98)
   java.lang.Thread.run(Unknown Source)

0.0% (127.8micros out of 500ms) cpu usage by thread 'elasticsearch[DeWolff, Jean][transport_client_timer][T#1]{Hashed wheel timer #1}'
 10/10 snapshots sharing following 5 elements
   java.lang.Thread.sleep(Native Method)
   org.elasticsearch.common.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:467)
   org.elasticsearch.common.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:376)
   org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
   java.lang.Thread.run(Unknown Source)

0.0% (51.4micros out of 500ms) cpu usage by thread 'JRubyWorker-8'
 10/10 snapshots sharing following 9 elements
   sun.misc.Unsafe.park(Native Method)
   java.util.concurrent.locks.LockSupport.park(Unknown Source)
   java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
   java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
   java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
   java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
   java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
   java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
   java.lang.Thread.run(Unknown Source)

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/44e50da7-89b1-415f-ae46-37a317f51e0d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

jprante · August 4, 2014, 5:31pm

Do you run many segments in your shards? If so, you should consider to run
optimize.

Jörg

On Mon, Aug 4, 2014 at 6:46 PM, Melanie Zamora mzamora@springcm.com wrote:

Hello,

My ES node is running very high and this is the thread from running
hot_threads. Can someone help me understand why CPU is so high?

This is a 1 node cluster running a linux machine with 16 cores and 72GB of
RAM.

::: [c9q9o027][VTde-tWsTcm9oe7vBZuadA][inet[/10.1.6.21:9300]]

48.3% (241.3ms out of 500ms) cpu usage by thread 'elasticsearch[c9q9o027][search][T#3]'
10/10 snapshots sharing following 17 elements
org.apache.lucene.search.ConstantScoreQuery$ConstantScorer.score(ConstantScoreQuery.java:245)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:627)
org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:167)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:491)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:281)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:269)
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:119)
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:233)
org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)
java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
java.lang.Thread.run(Unknown Source)

48.3% (241.3ms out of 500ms) cpu usage by thread 'elasticsearch[c9q9o027][search][T#8]'
2/10 snapshots sharing following 23 elements
org.apache.lucene.search.ConstantScoreQuery$ConstantScorer.docID(ConstantScoreQuery.java:190)
org.apache.lucene.search.DisjunctionScorer.heapAdjust(DisjunctionScorer.java:58)
org.apache.lucene.search.DisjunctionSumScorer.nextDoc(DisjunctionSumScorer.java:54)
org.elasticsearch.common.lucene.docset.DocIdSets.toFixedBitSet(DocIdSets.java:143)
org.elasticsearch.common.lucene.docset.DocIdSets.toSafeBits(DocIdSets.java:134)
org.elasticsearch.common.lucene.search.FilteredCollector.setNextReader(FilteredCollector.java:69)
org.elasticsearch.common.lucene.MultiCollector.setNextReader(MultiCollector.java:68)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:618)
org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:167)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:491)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:281)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:269)
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:119)
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:233)
org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)
java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
java.lang.Thread.run(Unknown Source)
8/10 snapshots sharing following 23 elements
org.apache.lucene.search.ConstantScoreQuery$ConstantScorer.docID(ConstantScoreQuery.java:190)
org.apache.lucene.search.DisjunctionSumScorer.nextDoc(DisjunctionSumScorer.java:61)
org.apache.lucene.search.DisjunctionSumScorer.nextDoc(DisjunctionSumScorer.java:53)
org.elasticsearch.common.lucene.docset.DocIdSets.toFixedBitSet(DocIdSets.java:143)
org.elasticsearch.common.lucene.docset.DocIdSets.toSafeBits(DocIdSets.java:134)
org.elasticsearch.common.lucene.search.FilteredCollector.setNextReader(FilteredCollector.java:69)
org.elasticsearch.common.lucene.MultiCollector.setNextReader(MultiCollector.java:68)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:618)
org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:167)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:491)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:281)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:269)
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:119)
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:233)
org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)
java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
java.lang.Thread.run(Unknown Source)

39.8% (199.2ms out of 500ms) cpu usage by thread 'elasticsearch[c9q9o027][search][T#15]'
10/10 snapshots sharing following 23 elements
org.elasticsearch.common.lucene.search.XBooleanFilter.getDocIdSet(XBooleanFilter.java:224)
org.elasticsearch.common.lucene.search.ApplyAcceptedDocsFilter.getDocIdSet(ApplyAcceptedDocsFilter.java:45)
org.apache.lucene.search.FilteredQuery$1.scorer(FilteredQuery.java:128)
org.apache.lucene.search.QueryWrapperFilter$1.iterator(QueryWrapperFilter.java:59)
org.elasticsearch.common.lucene.docset.DocIdSets.toSafeBits(DocIdSets.java:130)
org.elasticsearch.common.lucene.search.FilteredCollector.setNextReader(FilteredCollector.java:69)
org.elasticsearch.common.lucene.MultiCollector.setNextReader(MultiCollector.java:68)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:618)
org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:167)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:491)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:281)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:269)
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:119)
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:233)
org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)
java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
java.lang.Thread.run(Unknown Source)

::: [DeWolff, Jean][Z2ezcYq2QDiEwHjnbsLiYQ][inet[/10.1.6.21:9301]]{client=true, data=false}
0.9% (4.6ms out of 500ms) cpu usage by thread '<redis'
 10/10 snapshots sharing following 35 elements
   java.lang.Object.wait(Native Method)
   org.jruby.RubyThread.sleep(RubyThread.java:864)
   org.jruby.RubyKernel.sleep(RubyKernel.java:801)
   org.jruby.RubyKernel$INVOKER$s$0$1$sleep.call(RubyKernel$INVOKER$s$0$1$sleep.gen)
   org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:646)
   org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:205)
   org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:168)
   org.jruby.ast.FCallOneArgNode.interpret(FCallOneArgNode.java:36)
   org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
   org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
   org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
   org.jruby.ast.RescueBodyNode.interpret(RescueBodyNode.java:108)
   org.jruby.ast.RescueNode.handleException(RescueNode.java:177)
   org.jruby.ast.RescueNode.interpret(RescueNode.java:123)
   org.jruby.ast.BeginNode.interpret(BeginNode.java:83)
   org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
   org.jruby.ast.WhileNode.interpret(WhileNode.java:131)
   org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
   org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
   org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:75)
   org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:225)
   org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:204)
   org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:346)
   org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:204)
   org.jruby.ast.FCallSpecialArgNode.interpret(FCallSpecialArgNode.java:41)
   org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
   org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112)
   org.jruby.runtime.Interpreted19Block.evalBlockBody(Interpreted19Block.java:209)
   org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:197)
   org.jruby.runtime.Interpreted19Block.call(Interpreted19Block.java:128)
   org.jruby.runtime.Block.call(Block.java:89)
   org.jruby.RubyProc.call(RubyProc.java:261)
   org.jruby.RubyProc.call(RubyProc.java:213)
   org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:98)
   java.lang.Thread.run(Unknown Source)

0.0% (127.8micros out of 500ms) cpu usage by thread 'elasticsearch[DeWolff, Jean][transport_client_timer][T#1]{Hashed wheel timer #1}'
 10/10 snapshots sharing following 5 elements
   java.lang.Thread.sleep(Native Method)
   org.elasticsearch.common.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:467)
   org.elasticsearch.common.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:376)
   org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
   java.lang.Thread.run(Unknown Source)

0.0% (51.4micros out of 500ms) cpu usage by thread 'JRubyWorker-8'
 10/10 snapshots sharing following 9 elements
   sun.misc.Unsafe.park(Native Method)
   java.util.concurrent.locks.LockSupport.park(Unknown Source)
   java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
   java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
   java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
   java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
   java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
   java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
   java.lang.Thread.run(Unknown Source)
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/44e50da7-89b1-415f-ae46-37a317f51e0d%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/44e50da7-89b1-415f-ae46-37a317f51e0d%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoERT%3DHr_K9Budh%3DsY6ZQj%3DCzs3EONAjHnVyaD1RaniHYw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Topic		Replies	Views
ES High cpu issues Elasticsearch	11	1061	September 6, 2018
High CPU consumption Elasticsearch	8	8630	July 5, 2017
CPU utilization crossing 98% always Elasticsearch	2	1980	March 18, 2019
ES Server high CPU on CloseableThreadLocal Elasticsearch	4	1031	December 19, 2017
CPU utilization crossing 98% Elasticsearch	2	767	April 24, 2019

My Elasticsearch is running at very high CPU (constantly 99%) - Need help understanding hot_threads

Related topics