# Elasticsearch High CPU high load when searching

**URL:** <https://discuss.elastic.co/t/elasticsearch-high-cpu-high-load-when-searching/19618>\
**Category:** Elasticsearch\
**Created:** [September 4, 2014, 10:18am UTC](https://discuss.elastic.co/t/elasticsearch-high-cpu-high-load-when-searching/19618 "2014-09-04T10:18:00Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![11161](https://avatars.discourse-cdn.com/v4/letter/1/45deac/32.png) [@11161](https://discuss.elastic.co/u/11161)\
**Post date:** [September 4, 2014, 10:18am UTC](https://discuss.elastic.co/t/elasticsearch-high-cpu-high-load-when-searching/19618/1 "2014-09-04T10:18:00Z")

</div>

Hi,everyone  
I'm new to es ,i got a probem with es and tried a lot but still don't know  
how to change the situation ,so i would be gratitude if you can give me  
some advice about better performance of es searching

I build a es cluster with 2 nodes, in order to analyse logs for develop engineers  
,like some 5XX,4XX status and so on. It's running good ,but when i open  
kibana (only one user) which will do some queries to es, then the 2 servers  
will perform high cpu load. when i close kibana ,cpu load will be nomal  
again.

#### there are some details about the problem below:

1.marvel basic monitor  
[https://plus.google.com/107610728578662862458/posts/GM8jjjX1HTi](https://plus.google.com/107610728578662862458/posts/GM8jjjX1HTi)

2.gc monitor  
[https://plus.google.com/107610728578662862458/posts/PW7LCeDjemq](https://plus.google.com/107610728578662862458/posts/PW7LCeDjemq)

3.hot thread  
[https://docs.google.com/a/widget-inc.com/document/d/199CS0YHrAOWIbrNZssjpArOH9E0F11wHwIrO9XjmT-U/edit?usp=sharing](https://docs.google.com/a/widget-inc.com/document/d/199CS0YHrAOWIbrNZssjpArOH9E0F11wHwIrO9XjmT-U/edit?usp=sharing)

#### My server basic information:

RAM: 48G  
cpu cores :4  
centos 6.5 kernel version 3.8.13

jdk : jdk1.7.0\_67  
es version : 1.1.1

#### the work i have done:

when i google the problem, i got some possible solution

1.the leap-second  
since my kernel is 3.8.13, so the kernel had solved this problem

I'm sorry for my bad english.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/cb463ea7-88d2-417e-b3d9-07256c2e3b6a%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/cb463ea7-88d2-417e-b3d9-07256c2e3b6a%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [September 4, 2014, 10:28am UTC](https://discuss.elastic.co/t/elasticsearch-high-cpu-high-load-when-searching/19618/2 "2014-09-04T10:28:33Z")

</div>

Your second two links do not work.

Can you add another node, or close some old indexes?

Regards,  
Mark Walkom

Infrastructure Engineer  
Campaign Monitor  
email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com)  
web: [www.campaignmonitor.com](http://www.campaignmonitor.com)

On 4 September 2014 20:18, 王星龙 [xinglong.wang@upai.com](mailto:xinglong.wang@upai.com) wrote:

> Hi,everyone  
> I'm new to es ,i got a probem with es and tried a lot but still don't know  
> how to change the situation ,so i would be gratitude if you can give me  
> some advice about better performance of es searching
> 
> I build a es cluster with 2 nodes, in order to analyse logs for develop engineers  
> ,like some 5XX,4XX status and so on. It's running good ,but when i open  
> kibana (only one user) which will do some queries to es, then the 2 servers  
> will perform high cpu load. when i close kibana ,cpu load will be nomal  
> again.
> 
> #### there are some details about the problem below:
> 
> 1.marvel basic monitor  
> [https://plus.google.com/107610728578662862458/posts/GM8jjjX1HTi](https://plus.google.com/107610728578662862458/posts/GM8jjjX1HTi)
> 
> 2.gc monitor  
> [https://plus.google.com/107610728578662862458/posts/PW7LCeDjemq](https://plus.google.com/107610728578662862458/posts/PW7LCeDjemq)
> 
> 3.hot thread  
> [https://docs.google.com/a/widget-inc.com/document/d/199CS0YHrAOWIbrNZssjpArOH9E0F11wHwIrO9XjmT-U/edit?usp=sharing](https://docs.google.com/a/widget-inc.com/document/d/199CS0YHrAOWIbrNZssjpArOH9E0F11wHwIrO9XjmT-U/edit?usp=sharing)
> 
> #### My server basic information:
> 
> RAM: 48G  
> cpu cores :4  
> centos 6.5 kernel version 3.8.13
> 
> jdk : jdk1.7.0\_67  
> es version : 1.1.1
> 
> #### the work i have done:
> 
> when i google the problem, i got some possible solution
> 
> 1.the leap-second  
> since my kernel is 3.8.13, so the kernel had solved this problem
> 
> I'm sorry for my bad english.
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/cb463ea7-88d2-417e-b3d9-07256c2e3b6a%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/cb463ea7-88d2-417e-b3d9-07256c2e3b6a%40googlegroups.com)  
> [https://groups.google.com/d/msgid/elasticsearch/cb463ea7-88d2-417e-b3d9-07256c2e3b6a%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/cb463ea7-88d2-417e-b3d9-07256c2e3b6a%40googlegroups.com?utm_medium=email&utm_source=footer)  
> .  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624bWUfahJzfVqM4D3jw494kBDXp7wRJNEDNtAs7UtfFZrg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624bWUfahJzfVqM4D3jw494kBDXp7wRJNEDNtAs7UtfFZrg%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![11161](https://avatars.discourse-cdn.com/v4/letter/1/45deac/32.png) [@11161](https://discuss.elastic.co/u/11161)\
**Post date:** [September 4, 2014, 10:28am UTC](https://discuss.elastic.co/t/elasticsearch-high-cpu-high-load-when-searching/19618/3 "2014-09-04T10:28:50Z")

</div>

gc-monitor pic :

> **[New community features for Google Chat and an update on Currents](https://workspaceupdates.googleblog.com/2023/04/new-community-features-for-google-chat-and-an-update-currents%20.html)**
>
> Note:  This blog post outlines upcoming changes to Google Currents for Workspace users. For information on the previous deprecation of Googl...

hot-thread:

34.2% (171ms out of 500ms) cpu usage by thread  
'elasticsearch[node203][search][T#10]'

```
2/10 snapshots sharing following 23 elements

  org.apache.lucene.search.DisjunctionScorer.heapAdjust(DisjunctionScorer.java:66)

  org.apache.lucene.search.DisjunctionScorer.advance(DisjunctionScorer.java:150)

  org.apache.lucene.search.ReqExclScorer.toNonExcluded(ReqExclScorer.java:79)

  org.apache.lucene.search.ReqExclScorer.nextDoc(ReqExclScorer.java:58)

  org.apache.lucene.search.BooleanScorer2.nextDoc(BooleanScorer2.java:311)

  org.apache.lucene.search.DisjunctionScorer.nextDoc(DisjunctionScorer.java:130)

  org.elasticsearch.common.lucene.docset.DocIdSets.toFixedBitSet(DocIdSets.java:143)

  org.elasticsearch.common.lucene.docset.DocIdSets.toSafeBits(DocIdSets.java:134)

  org.elasticsearch.common.lucene.search.FilteredCollector.setNextReader(FilteredCollector.java:68)

  org.elasticsearch.common.lucene.MultiCollector.setNextReader(MultiCollector.java:68)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:612)

  org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:173)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)

  org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:111)

  org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:257)

  org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)

  org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)

  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

  java.lang.Thread.run(Thread.java:745)

2/10 snapshots sharing following 23 elements

  org.apache.lucene.search.DisjunctionScorer.heapAdjust(DisjunctionScorer.java:56)

  org.apache.lucene.search.DisjunctionScorer.advance(DisjunctionScorer.java:150)

  org.apache.lucene.search.ReqExclScorer.toNonExcluded(ReqExclScorer.java:79)

  org.apache.lucene.search.ReqExclScorer.nextDoc(ReqExclScorer.java:58)

  org.apache.lucene.search.BooleanScorer2.nextDoc(BooleanScorer2.java:311)

  org.apache.lucene.search.DisjunctionScorer.nextDoc(DisjunctionScorer.java:130)

  org.elasticsearch.common.lucene.docset.DocIdSets.toFixedBitSet(DocIdSets.java:143)

  org.elasticsearch.common.lucene.docset.DocIdSets.toSafeBits(DocIdSets.java:134)

  org.elasticsearch.common.lucene.search.FilteredCollector.setNextReader(FilteredCollector.java:68)

  org.elasticsearch.common.lucene.MultiCollector.setNextReader(MultiCollector.java:68)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:612)

  org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:173)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)

  org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:111)

  org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:257)

  org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)

  org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)

  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

  java.lang.Thread.run(Thread.java:745)

2/10 snapshots sharing following 20 elements

  org.apache.lucene.search.ReqExclScorer.nextDoc(ReqExclScorer.java:58)

  org.apache.lucene.search.BooleanScorer2.nextDoc(BooleanScorer2.java:311)

  org.apache.lucene.search.DisjunctionScorer.nextDoc(DisjunctionScorer.java:130)

  org.elasticsearch.common.lucene.docset.DocIdSets.toFixedBitSet(DocIdSets.java:143)

  org.elasticsearch.common.lucene.docset.DocIdSets.toSafeBits(DocIdSets.java:134)

  org.elasticsearch.common.lucene.search.FilteredCollector.setNextReader(FilteredCollector.java:68)

  org.elasticsearch.common.lucene.MultiCollector.setNextReader(MultiCollector.java:68)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:612)

  org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:173)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)

  org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:111)

  org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:257)

  org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)

  org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)

  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

  java.lang.Thread.run(Thread.java:745)

4/10 snapshots sharing following 19 elements

  org.apache.lucene.search.DisjunctionScorer.heapAdjust(DisjunctionScorer.java:61)

  org.apache.lucene.search.DisjunctionScorer.nextDoc(DisjunctionScorer.java:131)

  org.elasticsearch.common.lucene.docset.DocIdSets.toFixedBitSet(DocIdSets.java:143)

  org.elasticsearch.common.lucene.docset.DocIdSets.toSafeBits(DocIdSets.java:134)

  org.elasticsearch.common.lucene.search.FilteredCollector.setNextReader(FilteredCollector.java:68)

  org.elasticsearch.common.lucene.MultiCollector.setNextReader(MultiCollector.java:68)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:612)

  org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:173)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)

  org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:111)

  org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:257)

  org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)

  org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)

  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

  java.lang.Thread.run(Thread.java:745)

```

25.1% (125.7ms out of 500ms) cpu usage by thread  
'elasticsearch[node203][search][T#7]'

```
10/10 snapshots sharing following 16 elements

  org.apache.lucene.search.BooleanScorer.score(BooleanScorer.java:297)

  org.apache.lucene.search.BooleanScorer.score(BooleanScorer.java:339)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)

  org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:173)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:581)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:533)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:510)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:345)

  org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:116)

  org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:257)

  org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:623)

  org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:612)

  org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:270)

  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

  java.lang.Thread.run(Thread.java:745)

```

24.3% (121.6ms out of 500ms) cpu usage by thread  
'elasticsearch[node203][search][T#13]'

```
2/10 snapshots sharing following 23 elements

  org.apache.lucene.search.DisjunctionScorer.heapAdjust(DisjunctionScorer.java:61)

  org.apache.lucene.search.DisjunctionScorer.advance(DisjunctionScorer.java:150)

  org.apache.lucene.search.ReqExclScorer.toNonExcluded(ReqExclScorer.java:79)

  org.apache.lucene.search.ReqExclScorer.nextDoc(ReqExclScorer.java:58)

  org.apache.lucene.search.BooleanScorer2.nextDoc(BooleanScorer2.java:311)

  org.apache.lucene.search.DisjunctionScorer.nextDoc(DisjunctionScorer.java:130)

  org.elasticsearch.common.lucene.docset.DocIdSets.toFixedBitSet(DocIdSets.java:143)

  org.elasticsearch.common.lucene.docset.DocIdSets.toSafeBits(DocIdSets.java:134)

  org.elasticsearch.common.lucene.search.FilteredCollector.setNextReader(FilteredCollector.java:68)

  org.elasticsearch.common.lucene.MultiCollector.setNextReader(MultiCollector.java:68)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:612)

  org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:173)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)

  org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:111)

  org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:257)

  org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)

  org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)

  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

  java.lang.Thread.run(Thread.java:745)

3/10 snapshots sharing following 22 elements

  org.apache.lucene.search.DisjunctionScorer.advance(DisjunctionScorer.java:158)

  org.apache.lucene.search.ReqExclScorer.toNonExcluded(ReqExclScorer.java:79)

  org.apache.lucene.search.ReqExclScorer.nextDoc(ReqExclScorer.java:58)

  org.apache.lucene.search.BooleanScorer2.nextDoc(BooleanScorer2.java:311)

  org.apache.lucene.search.DisjunctionScorer.nextDoc(DisjunctionScorer.java:130)

  org.elasticsearch.common.lucene.docset.DocIdSets.toFixedBitSet(DocIdSets.java:143)

  org.elasticsearch.common.lucene.docset.DocIdSets.toSafeBits(DocIdSets.java:134)

  org.elasticsearch.common.lucene.search.FilteredCollector.setNextReader(FilteredCollector.java:68)

  org.elasticsearch.common.lucene.MultiCollector.setNextReader(MultiCollector.java:68)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:612)

  org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:173)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)

  org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:111)

  org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:257)

  org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)

  org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)

  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

  java.lang.Thread.run(Thread.java:745)

2/10 snapshots sharing following 24 elements

  org.apache.lucene.search.FilteredDocIdSetIterator.nextDoc(FilteredDocIdSetIterator.java:60)

  org.apache.lucene.search.ConstantScoreQuery$ConstantScorer.nextDoc(ConstantScoreQuery.java:196)

  org.apache.lucene.search.ConjunctionScorer.nextDoc(ConjunctionScorer.java:99)

  org.apache.lucene.search.ReqExclScorer.toNonExcluded(ReqExclScorer.java:88)

  org.apache.lucene.search.ReqExclScorer.nextDoc(ReqExclScorer.java:58)

  org.apache.lucene.search.BooleanScorer2.nextDoc(BooleanScorer2.java:311)

  org.apache.lucene.search.DisjunctionScorer.nextDoc(DisjunctionScorer.java:130)

  org.elasticsearch.common.lucene.docset.DocIdSets.toFixedBitSet(DocIdSets.java:143)

  org.elasticsearch.common.lucene.docset.DocIdSets.toSafeBits(DocIdSets.java:134)

  org.elasticsearch.common.lucene.search.FilteredCollector.setNextReader(FilteredCollector.java:68)

  org.elasticsearch.common.lucene.MultiCollector.setNextReader(MultiCollector.java:68)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:612)

  org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:173)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)

  org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:111)

  org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:257)

  org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)

  org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)

  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

  java.lang.Thread.run(Thread.java:745)

2/10 snapshots sharing following 19 elements

  org.apache.lucene.search.DisjunctionScorer.heapAdjust(DisjunctionScorer.java:61)

  org.apache.lucene.search.DisjunctionScorer.nextDoc(DisjunctionScorer.java:131)

  org.elasticsearch.common.lucene.docset.DocIdSets.toFixedBitSet(DocIdSets.java:143)

  org.elasticsearch.common.lucene.docset.DocIdSets.toSafeBits(DocIdSets.java:134)

  org.elasticsearch.common.lucene.search.FilteredCollector.setNextReader(FilteredCollector.java:68)

  org.elasticsearch.common.lucene.MultiCollector.setNextReader(MultiCollector.java:68)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:612)

  org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:173)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)

  org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:111)

  org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:257)

  org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)

  org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)

  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

  java.lang.Thread.run(Thread.java:745)

unique snapshot

  org.apache.lucene.search.DisjunctionScorer.heapAdjust(DisjunctionScorer.java:56)

  org.apache.lucene.search.DisjunctionScorer.nextDoc(DisjunctionScorer.java:131)

  org.elasticsearch.common.lucene.docset.DocIdSets.toFixedBitSet(DocIdSets.java:143)

  org.elasticsearch.common.lucene.docset.DocIdSets.toSafeBits(DocIdSets.java:134)

  org.elasticsearch.common.lucene.search.FilteredCollector.setNextReader(FilteredCollector.java:68)

  org.elasticsearch.common.lucene.MultiCollector.setNextReader(MultiCollector.java:68)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:612)

  org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:173)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)

  org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:111)

  org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:257)

  org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)

  org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)

  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

  java.lang.Thread.run(Thread.java:745)

```

:::  
[node202][XRa3t0ZzRl6eiKRwcKpLsg][UPWEB-PHP-043][inet[192.168.1.43/192.168.1.43:9300]]{master=true}

60.8% (303.9ms out of 500ms) cpu usage by thread  
'elasticsearch[node202][search][T#1]'

```
4/10 snapshots sharing following 16 elements

  org.elasticsearch.search.facet.terms.strings.TermsStringOrdinalsFacetExecutor$Collector.collect(TermsStringOrdinalsFacetExecutor.java:223)

  org.elasticsearch.common.lucene.search.FilteredCollector.collect(FilteredCollector.java:61)

  org.elasticsearch.common.lucene.MultiCollector.collect(MultiCollector.java:60)

  org.apache.lucene.search.Scorer.score(Scorer.java:65)

  org.apache.lucene.search.ConstantScoreQuery$ConstantScorer.score(ConstantScoreQuery.java:256)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)

  org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:173)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)

  org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:111)

  org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:257)

  org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:623)

  org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:612)

  org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:270)

  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

  java.lang.Thread.run(Thread.java:745)

2/10 snapshots sharing following 14 elements

  org.elasticsearch.common.lucene.MultiCollector.collect(MultiCollector.java:60)

  org.apache.lucene.search.Scorer.score(Scorer.java:65)

  org.apache.lucene.search.ConstantScoreQuery$ConstantScorer.score(ConstantScoreQuery.java:256)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)

  org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:173)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)

  org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:111)

  org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:257)

  org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:623)

  org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:612)

  org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:270)

  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

  java.lang.Thread.run(Thread.java:745)

4/10 snapshots sharing following 14 elements

  org.apache.lucene.search.ConstantScoreQuery$ConstantScorer.nextDoc(ConstantScoreQuery.java:196)

  org.apache.lucene.search.Scorer.score(Scorer.java:64)

  org.apache.lucene.search.ConstantScoreQuery$ConstantScorer.score(ConstantScoreQuery.java:256)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)

  org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:173)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)

  org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:111)

  org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:257)

  org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:623)

  org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:612)

  org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:270)

  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

  java.lang.Thread.run(Thread.java:745)

```

49.0% (245ms out of 500ms) cpu usage by thread  
'elasticsearch[node202][search][T#20]'

```
2/10 snapshots sharing following 23 elements

  org.apache.lucene.search.DisjunctionScorer.heapAdjust(DisjunctionScorer.java:56)

  org.apache.lucene.search.DisjunctionScorer.advance(DisjunctionScorer.java:150)

  org.apache.lucene.search.ReqExclScorer.toNonExcluded(ReqExclScorer.java:79)

  org.apache.lucene.search.ReqExclScorer.advance(ReqExclScorer.java:129)

  org.apache.lucene.search.BooleanScorer2.advance(BooleanScorer2.java:328)

  org.apache.lucene.search.DisjunctionScorer.advance(DisjunctionScorer.java:149)

  org.apache.lucene.search.FilteredQuery$LeapFrogScorer.score(FilteredQuery.java:256)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)

  org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:173)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:581)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:533)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:510)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:345)

  org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:116)

  org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:257)

  org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)

  org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)

  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

  java.lang.Thread.run(Thread.java:745)

8/10 snapshots sharing following 17 elements

  org.apache.lucene.search.FilteredQuery$LeapFrogScorer.score(FilteredQuery.java:256)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)

  org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:173)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:581)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:533)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:510)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:345)

  org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:116)

  org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:257)

  org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)

  org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)

  org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)

  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

  java.lang.Thread.run(Thread.java:745)

```

45.6% (228.1ms out of 500ms) cpu usage by thread  
'elasticsearch[node202][search][T#16]'

```
4/10 snapshots sharing following 16 elements

  org.apache.lucene.search.DisjunctionScorer.nextDoc(DisjunctionScorer.java:131)

  org.elasticsearch.common.lucene.docset.DocIdSets.toFixedBitSet(DocIdSets.java:143)

  org.elasticsearch.common.lucene.docset.DocIdSets.toSafeBits(DocIdSets.java:134)

  org.elasticsearch.common.lucene.search.FilteredCollector.setNextReader(FilteredCollector.java:68)

  org.elasticsearch.common.lucene.MultiCollector.setNextReader(MultiCollector.java:68)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:612)

  org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:173)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)

  org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:111)

  org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:257)

  org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:623)

  org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:612)

  org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:270)

  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

  java.lang.Thread.run(Thread.java:745)

2/10 snapshots sharing following 19 elements

  org.apache.lucene.search.ConstantScoreQuery$ConstantScorer.docID(ConstantScoreQuery.java:201)

  org.apache.lucene.search.DisjunctionSumScorer.countMatches(DisjunctionSumScorer.java:64)

  org.apache.lucene.search.DisjunctionSumScorer.afterNext(DisjunctionSumScorer.java:55)

  org.apache.lucene.search.DisjunctionScorer.nextDoc(DisjunctionScorer.java:139)

  org.elasticsearch.common.lucene.docset.DocIdSets.toFixedBitSet(DocIdSets.java:143)

  org.elasticsearch.common.lucene.docset.DocIdSets.toSafeBits(DocIdSets.java:134)

  org.elasticsearch.common.lucene.search.FilteredCollector.setNextReader(FilteredCollector.java:68)

  org.elasticsearch.common.lucene.MultiCollector.setNextReader(MultiCollector.java:68)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:612)

  org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:173)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)

  org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:111)

  org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:257)

  org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:623)

  org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:612)

  org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:270)

  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

  java.lang.Thread.run(Thread.java:745)

2/10 snapshots sharing following 17 elements

  org.apache.lucene.search.ConstantScoreQuery$ConstantScorer.docID(ConstantScoreQuery.java:201)

  org.apache.lucene.search.DisjunctionScorer.nextDoc(DisjunctionScorer.java:138)

  org.elasticsearch.common.lucene.docset.DocIdSets.toFixedBitSet(DocIdSets.java:143)

  org.elasticsearch.common.lucene.docset.DocIdSets.toSafeBits(DocIdSets.java:134)

  org.elasticsearch.common.lucene.search.FilteredCollector.setNextReader(FilteredCollector.java:68)

  org.elasticsearch.common.lucene.MultiCollector.setNextReader(MultiCollector.java:68)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:612)

  org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:173)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)

  org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:111)

  org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:257)

  org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:623)

  org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:612)

  org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:270)

  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

  java.lang.Thread.run(Thread.java:745)

2/10 snapshots sharing following 17 elements

  org.apache.lucene.search.DisjunctionSumScorer.afterNext(DisjunctionSumScorer.java:50)

  org.apache.lucene.search.DisjunctionScorer.nextDoc(DisjunctionScorer.java:139)

  org.elasticsearch.common.lucene.docset.DocIdSets.toFixedBitSet(DocIdSets.java:143)

  org.elasticsearch.common.lucene.docset.DocIdSets.toSafeBits(DocIdSets.java:134)

  org.elasticsearch.common.lucene.search.FilteredCollector.setNextReader(FilteredCollector.java:68)

  org.elasticsearch.common.lucene.MultiCollector.setNextReader(MultiCollector.java:68)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:612)

  org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:173)

  org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)

  org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:111)

  org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:257)

  org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:623)

  org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:612)

  org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:270)

  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

  java.lang.Thread.run(Thread.java:745)

```

在 2014年9月4日星期四UTC+8下午6时18分01秒，王星龙写道：

> Hi,everyone  
> I'm new to es ,i got a probem with es and tried a lot but still don't know  
> how to change the situation ,so i would be gratitude if you can give me  
> some advice about better performance of es searching
> 
> I build a es cluster with 2 nodes, in order to analyse logs for develop engineers  
> ,like some 5XX,4XX status and so on. It's running good ,but when i open  
> kibana (only one user) which will do some queries to es, then the 2 servers  
> will perform high cpu load. when i close kibana ,cpu load will be nomal  
> again.
> 
> #### there are some details about the problem below:
> 
> 1.marvel basic monitor  
> [https://plus.google.com/107610728578662862458/posts/GM8jjjX1HTi](https://plus.google.com/107610728578662862458/posts/GM8jjjX1HTi)
> 
> 2.gc monitor  
> [https://plus.google.com/107610728578662862458/posts/PW7LCeDjemq](https://plus.google.com/107610728578662862458/posts/PW7LCeDjemq)
> 
> 3.hot thread  
> [https://docs.google.com/a/widget-inc.com/document/d/199CS0YHrAOWIbrNZssjpArOH9E0F11wHwIrO9XjmT-U/edit?usp=sharing](https://docs.google.com/a/widget-inc.com/document/d/199CS0YHrAOWIbrNZssjpArOH9E0F11wHwIrO9XjmT-U/edit?usp=sharing)
> 
> #### My server basic information:
> 
> RAM: 48G  
> cpu cores :4  
> centos 6.5 kernel version 3.8.13
> 
> jdk : jdk1.7.0\_67  
> es version : 1.1.1
> 
> #### the work i have done:
> 
> when i google the problem, i got some possible solution
> 
> 1.the leap-second  
> since my kernel is 3.8.13, so the kernel had solved this problem
> 
> I'm sorry for my bad english.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/5a1f7bcc-af2a-4639-bd52-b52b36445c55%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/5a1f7bcc-af2a-4639-bd52-b52b36445c55%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![11161](https://avatars.discourse-cdn.com/v4/letter/1/45deac/32.png) [@11161](https://discuss.elastic.co/u/11161)\
**Post date:** [September 4, 2014, 10:32am UTC](https://discuss.elastic.co/t/elasticsearch-high-cpu-high-load-when-searching/19618/4 "2014-09-04T10:32:09Z")

</div>

i just add the link of the second pic

在 2014年9月4日星期四UTC+8下午6时29分06秒，Mark Walkom写道：

> Your second two links do not work.
> 
> Can you add another node, or close some old indexes?
> 
> Regards,  
> Mark Walkom
> 
> Infrastructure Engineer  
> Campaign Monitor  
> email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com) \<javascript:\>  
> web: [www.campaignmonitor.com](http://www.campaignmonitor.com)
> 
> On 4 September 2014 20:18, 王星龙 \<[xinglo...@upai.com](mailto:xinglo...@upai.com) \<javascript:\>\> wrote:
> 
> > Hi,everyone  
> > I'm new to es ,i got a probem with es and tried a lot but still don't  
> > know how to change the situation ,so i would be gratitude if you can  
> > give me some advice about better performance of es searching
> > 
> > I build a es cluster with 2 nodes, in order to analyse logs for develop engineers  
> > ,like some 5XX,4XX status and so on. It's running good ,but when i open  
> > kibana (only one user) which will do some queries to es, then the 2 servers  
> > will perform high cpu load. when i close kibana ,cpu load will be nomal  
> > again.
> > 
> > #### there are some details about the problem below:
> > 
> > 1.marvel basic monitor  
> > [https://plus.google.com/107610728578662862458/posts/GM8jjjX1HTi](https://plus.google.com/107610728578662862458/posts/GM8jjjX1HTi)
> > 
> > 2.gc monitor  
> > [https://plus.google.com/107610728578662862458/posts/PW7LCeDjemq](https://plus.google.com/107610728578662862458/posts/PW7LCeDjemq)
> > 
> > 3.hot thread  
> > [https://docs.google.com/a/widget-inc.com/document/d/199CS0YHrAOWIbrNZssjpArOH9E0F11wHwIrO9XjmT-U/edit?usp=sharing](https://docs.google.com/a/widget-inc.com/document/d/199CS0YHrAOWIbrNZssjpArOH9E0F11wHwIrO9XjmT-U/edit?usp=sharing)
> > 
> > #### My server basic information:
> > 
> > RAM: 48G  
> > cpu cores :4  
> > centos 6.5 kernel version 3.8.13
> > 
> > jdk : jdk1.7.0\_67  
> > es version : 1.1.1
> > 
> > #### the work i have done:
> > 
> > when i google the problem, i got some possible solution
> > 
> > 1.the leap-second  
> > since my kernel is 3.8.13, so the kernel had solved this problem
> > 
> > I'm sorry for my bad english.
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \<javascript:\>.  
> > To view this discussion on the web visit  
> > [https://groups.google.com/d/msgid/elasticsearch/cb463ea7-88d2-417e-b3d9-07256c2e3b6a%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/cb463ea7-88d2-417e-b3d9-07256c2e3b6a%40googlegroups.com)  
> > [https://groups.google.com/d/msgid/elasticsearch/cb463ea7-88d2-417e-b3d9-07256c2e3b6a%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/cb463ea7-88d2-417e-b3d9-07256c2e3b6a%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > .  
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/80aa4dd9-9452-467d-9731-ae09b97018dd%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/80aa4dd9-9452-467d-9731-ae09b97018dd%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![11161](https://avatars.discourse-cdn.com/v4/letter/1/45deac/32.png) [@11161](https://discuss.elastic.co/u/11161)\
**Post date:** [September 4, 2014, 10:38am UTC](https://discuss.elastic.co/t/elasticsearch-high-cpu-high-load-when-searching/19618/5 "2014-09-04T10:38:40Z")

</div>

I just add the link of the second pic .

yes ,i can add more node to my cluster, but i want to know does i can do  
something to improve the performance, like gc or something else ?

what do you mean by "close the old indexes" ? do you means to delete the  
old indexs? if so ,does the old indexes influence the search ? because we  
always search latest log (today's log), am i right?

在 2014年9月4日星期四UTC+8下午6时29分06秒，Mark Walkom写道：

> Your second two links do not work.
> 
> Can you add another node, or close some old indexes?
> 
> Regards,  
> Mark Walkom
> 
> Infrastructure Engineer  
> Campaign Monitor  
> email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com) \<javascript:\>  
> web: [www.campaignmonitor.com](http://www.campaignmonitor.com)
> 
> On 4 September 2014 20:18, 王星龙 \<[xinglo...@upai.com](mailto:xinglo...@upai.com) \<javascript:\>\> wrote:
> 
> > Hi,everyone  
> > I'm new to es ,i got a probem with es and tried a lot but still don't  
> > know how to change the situation ,so i would be gratitude if you can  
> > give me some advice about better performance of es searching
> > 
> > I build a es cluster with 2 nodes, in order to analyse logs for develop engineers  
> > ,like some 5XX,4XX status and so on. It's running good ,but when i open  
> > kibana (only one user) which will do some queries to es, then the 2 servers  
> > will perform high cpu load. when i close kibana ,cpu load will be nomal  
> > again.
> > 
> > #### there are some details about the problem below:
> > 
> > 1.marvel basic monitor  
> > [https://plus.google.com/107610728578662862458/posts/GM8jjjX1HTi](https://plus.google.com/107610728578662862458/posts/GM8jjjX1HTi)
> > 
> > 2.gc monitor  
> > [https://plus.google.com/107610728578662862458/posts/PW7LCeDjemq](https://plus.google.com/107610728578662862458/posts/PW7LCeDjemq)
> > 
> > 3.hot thread  
> > [https://docs.google.com/a/widget-inc.com/document/d/199CS0YHrAOWIbrNZssjpArOH9E0F11wHwIrO9XjmT-U/edit?usp=sharing](https://docs.google.com/a/widget-inc.com/document/d/199CS0YHrAOWIbrNZssjpArOH9E0F11wHwIrO9XjmT-U/edit?usp=sharing)
> > 
> > #### My server basic information:
> > 
> > RAM: 48G  
> > cpu cores :4  
> > centos 6.5 kernel version 3.8.13
> > 
> > jdk : jdk1.7.0\_67  
> > es version : 1.1.1
> > 
> > #### the work i have done:
> > 
> > when i google the problem, i got some possible solution
> > 
> > 1.the leap-second  
> > since my kernel is 3.8.13, so the kernel had solved this problem
> > 
> > I'm sorry for my bad english.
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \<javascript:\>.  
> > To view this discussion on the web visit  
> > [https://groups.google.com/d/msgid/elasticsearch/cb463ea7-88d2-417e-b3d9-07256c2e3b6a%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/cb463ea7-88d2-417e-b3d9-07256c2e3b6a%40googlegroups.com)  
> > [https://groups.google.com/d/msgid/elasticsearch/cb463ea7-88d2-417e-b3d9-07256c2e3b6a%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/cb463ea7-88d2-417e-b3d9-07256c2e3b6a%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > .  
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/4a602b19-c4f8-4feb-bd1a-9d2fda627bef%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/4a602b19-c4f8-4feb-bd1a-9d2fda627bef%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![11161](https://avatars.discourse-cdn.com/v4/letter/1/45deac/32.png) [@11161](https://discuss.elastic.co/u/11161)\
**Post date:** [September 5, 2014, 1:32am UTC](https://discuss.elastic.co/t/elasticsearch-high-cpu-high-load-when-searching/19618/6 "2014-09-05T01:32:14Z")

</div>

I want to add cache for es request, but this function is coming to 1.4.0

在 2014年9月4日星期四UTC+8下午6时29分06秒，Mark Walkom写道：

> Your second two links do not work.
> 
> Can you add another node, or close some old indexes?
> 
> Regards,  
> Mark Walkom
> 
> Infrastructure Engineer  
> Campaign Monitor  
> email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com) \<javascript:\>  
> web: [www.campaignmonitor.com](http://www.campaignmonitor.com)
> 
> On 4 September 2014 20:18, 王星龙 \<[xinglo...@upai.com](mailto:xinglo...@upai.com) \<javascript:\>\> wrote:
> 
> > Hi,everyone  
> > I'm new to es ,i got a probem with es and tried a lot but still don't  
> > know how to change the situation ,so i would be gratitude if you can  
> > give me some advice about better performance of es searching
> > 
> > I build a es cluster with 2 nodes, in order to analyse logs for develop engineers  
> > ,like some 5XX,4XX status and so on. It's running good ,but when i open  
> > kibana (only one user) which will do some queries to es, then the 2 servers  
> > will perform high cpu load. when i close kibana ,cpu load will be nomal  
> > again.
> > 
> > #### there are some details about the problem below:
> > 
> > 1.marvel basic monitor  
> > [https://plus.google.com/107610728578662862458/posts/GM8jjjX1HTi](https://plus.google.com/107610728578662862458/posts/GM8jjjX1HTi)
> > 
> > 2.gc monitor  
> > [https://plus.google.com/107610728578662862458/posts/PW7LCeDjemq](https://plus.google.com/107610728578662862458/posts/PW7LCeDjemq)
> > 
> > 3.hot thread  
> > [https://docs.google.com/a/widget-inc.com/document/d/199CS0YHrAOWIbrNZssjpArOH9E0F11wHwIrO9XjmT-U/edit?usp=sharing](https://docs.google.com/a/widget-inc.com/document/d/199CS0YHrAOWIbrNZssjpArOH9E0F11wHwIrO9XjmT-U/edit?usp=sharing)
> > 
> > #### My server basic information:
> > 
> > RAM: 48G  
> > cpu cores :4  
> > centos 6.5 kernel version 3.8.13
> > 
> > jdk : jdk1.7.0\_67  
> > es version : 1.1.1
> > 
> > #### the work i have done:
> > 
> > when i google the problem, i got some possible solution
> > 
> > 1.the leap-second  
> > since my kernel is 3.8.13, so the kernel had solved this problem
> > 
> > I'm sorry for my bad english.
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \<javascript:\>.  
> > To view this discussion on the web visit  
> > [https://groups.google.com/d/msgid/elasticsearch/cb463ea7-88d2-417e-b3d9-07256c2e3b6a%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/cb463ea7-88d2-417e-b3d9-07256c2e3b6a%40googlegroups.com)  
> > [https://groups.google.com/d/msgid/elasticsearch/cb463ea7-88d2-417e-b3d9-07256c2e3b6a%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/cb463ea7-88d2-417e-b3d9-07256c2e3b6a%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > .  
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/9dcc63a6-62aa-4691-9ca2-8493b9b8e450%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/9dcc63a6-62aa-4691-9ca2-8493b9b8e450%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:04am UTC](https://discuss.elastic.co/t/elasticsearch-high-cpu-high-load-when-searching/19618/7 "2017-07-06T01:04:14Z")

</div>