Elastic Search CPU Utilization.. on one cluster node Version1.7.2

image

Hi Team,

I am observing that there is a high CPU utilization on one my culster server. This has been regular.. I am helpless to my team.. I request you to please let me know what i need to do on this regards..

I restart elastic service on that node but no use.. I see the error as below when i check for hotthreads

::: [Prod-Node-Three-25][zP4lrdagQeSWxm6204N30A][pro-phvapp04][inet[/172.23.12.25:9300]]
Hot threads at 2019-04-25T05:28:52.758Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:

103.1% (515.6ms out of 500ms) cpu usage by thread 'elasticsearch[Prod-Node-Three-25][search][T#6]'
5/10 snapshots sharing following 14 elements
org.apache.lucene.codecs.blocktree.IntersectTermsEnum.next(IntersectTermsEnum.java:454)
org.apache.lucene.search.MultiTermQueryWrapperFilter.getDocIdSet(MultiTermQueryWrapperFilter.java:114)
org.apache.lucene.search.ConstantScoreQuery$ConstantWeight.scorer(ConstantScoreQuery.java:157)
org.apache.lucene.search.QueryWrapperFilter$1.iterator(QueryWrapperFilter.java:59)
org.elasticsearch.search.fetch.matchedqueries.MatchedQueriesFetchSubPhase.addMatchedQueries(MatchedQueriesFetchSubPhase.java:123)
org.elasticsearch.search.fetch.matchedqueries.MatchedQueriesFetchSubPhase.hitExecute(MatchedQueriesFetchSubPhase.java:80)
org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:194)
org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:516)
org.elasticsearch.search.action.SearchServiceTransportAction$17.call(SearchServiceTransportAction.java:452)
org.elasticsearch.search.action.SearchServiceTransportAction$17.call(SearchServiceTransportAction.java:449)
org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:559)
java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
java.lang.Thread.run(Unknown Source)
5/10 snapshots sharing following 9 elements
org.elasticsearch.search.fetch.matchedqueries.MatchedQueriesFetchSubPhase.hitExecute(MatchedQueriesFetchSubPhase.java:80)
org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:194)
org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:516)
org.elasticsearch.search.action.SearchServiceTransportAction$17.call(SearchServiceTransportAction.java:452)
org.elasticsearch.search.action.SearchServiceTransportAction$17.call(SearchServiceTransportAction.java:449)
org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:559)
java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
java.lang.Thread.run(Unknown Source)

It looks like there as a potentially expensive query running. How many nodes do you have in the cluster?

1.7.2 is also a very old version, which has been End-Of-Life for a very long time. I would therefore recommend you consider upgrading.

Thank you for the support. We will take it for the upgrade of this to latest version..
It has 2 nodes.

The node which consumes CPU is always a child node. Because of this consumption, The queries are effected on the master node.

How can i query, What expanse queries got executed?.

Till we upgrade it, Can you suggest us the alternative/remedy.

Many Thanks,
Kamesh

You might try enabling and tuning the slow log, but I have not used version 1.7 in years so have not got any other suggestions.

Thank you Christian for the quick response
Will check on you suggestion

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.