The elasticsearch process in prod currently has high CPU utilization. Would you help me understand what is going on?
http://sarvername:9200/_plugins/head/
Returns 404
http://sarvername:9200/_cluster/health?pretty=true
{
"cluster_name" : "web-elasticsearch-prod",
"status" : "red",
"timed_out" : false,
"number_of_nodes" : 1,
"number_of_data_nodes" : 1,
"active_primary_shards" : 7331,
"active_shards" : 7331,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 7341,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0
}
There is also a lot of these errors in the log file.
[2018-06-04 13:48:35,973][DEBUG][action.admin.indices.stats] [servername] [applications-uat-2017.05.29][4], node[X_eynmeNSqaktfC6FxPR-Q], [P], s[STARTED]: failed to execute [org.elasticsearch.action.admin.indices.stats.IndicesStatsRequest@79dcef21]
org.elasticsearch.ElasticsearchException: failed to refresh store stats
at org.elasticsearch.index.store.Store$StoreStatsCache.refresh(Store.java:1573)
at org.elasticsearch.index.store.Store$StoreStatsCache.refresh(Store.java:1558)
at org.elasticsearch.common.util.SingleObjectCache.getOrRefresh(SingleObjectCache.java:55)
at org.elasticsearch.index.store.Store.stats(Store.java:290)
at org.elasticsearch.index.shard.IndexShard.storeStats(IndexShard.java:639)
at org.elasticsearch.action.admin.indices.stats.CommonStats.<init>(CommonStats.java:139)
at org.elasticsearch.action.admin.indices.stats.ShardStats.<init>(ShardStats.java:55)
at org.elasticsearch.action.admin.indices.stats.TransportIndicesStatsAction.shardOperation(TransportIndicesStatsAction.java:208)
at org.elasticsearch.action.admin.indices.stats.TransportIndicesStatsAction.shardOperation(TransportIndicesStatsAction.java:56)
at org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$AsyncBroadcastAction$1.run(TransportBroadcastOperationAction.java:170)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)