# High CPU load, many hot threads with the same stack

**URL:** <https://discuss.elastic.co/t/high-cpu-load-many-hot-threads-with-the-same-stack/76271>\
**Category:** Elasticsearch\
**Created:** [February 23, 2017, 5:10pm UTC](https://discuss.elastic.co/t/high-cpu-load-many-hot-threads-with-the-same-stack/76271 "2017-02-23T17:10:49Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![jazz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jazz/32/6943_2.png) [@jazz](https://discuss.elastic.co/u/jazz)\
**Post date:** [February 23, 2017, 5:10pm UTC](https://discuss.elastic.co/t/high-cpu-load-many-hot-threads-with-the-same-stack/76271/1 "2017-02-23T17:10:49Z")

</div>

I have an Elasticsearch cluster running very slow sometimes (\>30s for each query) does to high CPU usage. In the hot threads, the thread I see the most is this one:

```auto
       sun.misc.Unsafe.park(Native Method)
       java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
       java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
       java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
       java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
       com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:285)
       com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
       com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:137)
       com.google.common.cache.LocalCache$LoadingValueReference.waitForValue(LocalCache.java:3557)
       com.google.common.cache.LocalCache$Segment.waitForLoadingValue(LocalCache.java:2302)
       com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2191)
       com.google.common.cache.LocalCache.get(LocalCache.java:3937)
       com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4739)
       org.elasticsearch.indices.fielddata.cache.IndicesFieldDataCache$IndexFieldCache.load(IndicesFieldDataCache.java:183)
       org.elasticsearch.index.fielddata.plain.SortedSetDVOrdinalsIndexFieldData.loadGlobal(SortedSetDVOrdinalsIndexFieldData.java:91)
       org.elasticsearch.search.aggregations.support.ValuesSource$Bytes$WithOrdinals$FieldData.globalOrdinalsValues(ValuesSource.java:144)
       org.elasticsearch.search.aggregations.support.ValuesSource$Bytes$WithOrdinals.globalMaxOrd(ValuesSource.java:117)
       org.elasticsearch.search.aggregations.bucket.terms.TermsAggregatorFactory.doCreateInternal(TermsAggregatorFactory.java:217)
       org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory.createInternal(ValuesSourceAggregatorFactory.java:64)
       org.elasticsearch.search.aggregations.AggregatorFactory.create(AggregatorFactory.java:102)
       org.elasticsearch.search.aggregations.AggregatorFactories.createTopLevelAggregators(AggregatorFactories.java:87)
       org.elasticsearch.search.aggregations.AggregationPhase.preProcess(AggregationPhase.java:85)
       org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:111)
       org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:372)
       org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:385)
       org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:368)
       org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:365)
       org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:33)
       org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:77)
       org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.doRun(MessageChannelHandler.java:293)
       org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
       java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
       java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
       java.lang.Thread.run(Thread.java:745)

```

How can I find the queries generating high CPU load, and how could I optimize it?

---

<div class="post-metadata">

**Author:** ![jkuang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jkuang/32/72637_2.png) [@jkuang](https://discuss.elastic.co/u/jkuang)\
**Post date:** [February 23, 2017, 8:10pm UTC](https://discuss.elastic.co/t/high-cpu-load-many-hot-threads-with-the-same-stack/76271/2 "2017-02-23T20:10:09Z")

</div>

We can match the time this hot thread appears with the time a query is executed against ES. I find the discussion below to be very helpful.

> <https://stackoverflow.com/questions/21749997/see-all-executed-elasticsearch-queries>

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 23, 2017, 8:10pm UTC](https://discuss.elastic.co/t/high-cpu-load-many-hot-threads-with-the-same-stack/76271/3 "2017-03-23T20:10:34Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.