ES1.7.2 - High CPU usage

[saihariharan (Hariharan M)]

We are facing high CPU usage on a lone node in a 5 node ES 1.7.2 cluster. All the cluster is currently doing is replicating/sync'ng documents from a Couchbase cluster through the transport-couchbase plugin.

CPU is around 80% on a single node. The remaining 4 nodes do not have this issue.

Upon analyzing the thread dump the following code seems to be causing the issue

"elasticsearch[Henry Peter Gyrich][search][T#35]" #11890 daemon prio=5 os_prio=0 tid=0x00007f938811e800 nid=0x268f runnable [0x00007f9379bdb000]
   java.lang.Thread.State: RUNNABLE
        at org.elasticsearch.common.lucene.docset.AndDocIdSet$AndBits.get(AndDocIdSet.java:116)
        at org.elasticsearch.common.lucene.docset.BitsDocIdSetIterator.matchDoc(BitsDocIdSetIterator.java:45)
        at org.elasticsearch.common.lucene.docset.MatchDocIdSetIterator.nextDoc(MatchDocIdSetIterator.java:50)
        at org.apache.lucene.search.FilteredDocIdSetIterator.nextDoc(FilteredDocIdSetIterator.java:59)
        at org.apache.lucene.search.ConstantScoreQuery$ConstantScorer.nextDoc(ConstantScoreQuery.java:257)
        at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:192)
        at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:163)
        at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)
        at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)
        at org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:191)
        at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)
        at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:117)
        at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:301)
        at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:312)
        at org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:776)
        at org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:767)
        at org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.doRun(MessageChannelHandler.java:279)
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:36)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

I checked another post with similar issue on the same ES version. However that relates to User queries on ES and nothing related to document indexing. Has anybody had similar issues or have any suggestions?

[spinscale (Alexander Reelsen)]

it is the same in this case, something/someone is executing a search on your cluster, which takes some CPU. However you left out the part, where it is shown how much percent are actually used for this (it might be very low and not significant).

I guess you heard this before, but Elasticsearch 1.7 is not being fixed/patched since a few years, so I really ought you to upgrade for a faster/more stable/bugfree version of Elasticsearch