# Continuous high CPU and load + EsRejectedExecutionException

**URL:** <https://discuss.elastic.co/t/continuous-high-cpu-and-load-esrejectedexecutionexception/151905>\
**Category:** Elasticsearch\
**Created:** [October 10, 2018, 5:49pm UTC](https://discuss.elastic.co/t/continuous-high-cpu-and-load-esrejectedexecutionexception/151905 "2018-10-10T17:49:30Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Roni\_T](https://avatars.discourse-cdn.com/v4/letter/r/d2c977/32.png) [@Roni\_T](https://discuss.elastic.co/u/Roni_T)\
**Post date:** [October 10, 2018, 5:49pm UTC](https://discuss.elastic.co/t/continuous-high-cpu-and-load-esrejectedexecutionexception/151905/1 "2018-10-10T17:49:30Z")

</div>

Hi all,

We're running a cluster with the following configuration:

- 6 data nodes (each has 31G RAM allocated to ES heap, and 16 vCPU - m4.4xlarge on AWS)
- 1 main index (others are very small)
- 21 shards
- 1 replica
- ~600,000,000 documents, and;
- 800G of data

We are running thousands of queries per second and continuously experiencing high CPU & load (almost 100%) causing EsRejectedExecutionException every few minutes.

We are using ES v2.3.4.

What would you recommend to do in this case (without modifying the ES version or increasing the number of indices)?

Would increasing the CPU or the number of nodes be helpful?

Many thanks!

---

<div class="post-metadata">

**Author:** ![dakrone](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dakrone/32/23351_2.png) [@dakrone](https://discuss.elastic.co/u/dakrone)\
**Post date:** [October 11, 2018, 8:22am UTC](https://discuss.elastic.co/t/continuous-high-cpu-and-load-esrejectedexecutionexception/151905/2 "2018-10-11T08:22:09Z")

</div>

Hi Roni,

I would recommend checking out the search slowlog to see which queries are taking a long time, from there, I think you may want to see if you can optimize those so that they don't take as long to execute in the queue.

You could also check the hot\_threads API: [Nodes hot\_threads | Elasticsearch Guide [6.4] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/6.4/cluster-nodes-hot-threads.html) to see what is taking the bulk of the processing.

> Would increasing the CPU or the number of nodes be helpful?

This would _probably_ help (I can't tell for certain without knowing exactly why the queries are filling the queue), so if you need a faster solution, you can almost always add resources to help with it.

Another thing I would recommend is to try and upgrade if you can, we've made a lot of improvements since 2.3.4, in particular, one that might help you is Adaptive Replica Selection: [Search APIs | Elasticsearch Guide [6.4] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/6.4/search.html#search-adaptive-replica)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 8, 2018, 8:22am UTC](https://discuss.elastic.co/t/continuous-high-cpu-and-load-esrejectedexecutionexception/151905/3 "2018-11-08T08:22:10Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.