# Increasing thread pool / queue size **URL:** \ **Category:** Elasticsearch\ **Created:** [May 18, 2014, 4:59pm UTC](https://discuss.elastic.co/t/increasing-thread-pool-queue-size/17579 "2014-05-18T16:59:36Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![jcm112388](https://avatars.discourse-cdn.com/v4/letter/j/b5e925/32.png) [@jcm112388](https://discuss.elastic.co/u/jcm112388)\ **Post date:** [May 18, 2014, 4:59pm UTC](https://discuss.elastic.co/t/increasing-thread-pool-queue-size/17579/1 "2014-05-18T16:59:36Z")
We're running v1.1.0 in production right now. We have 15 nodes and 15 shards w/ 4 replicas (75 total). Recently we've ran into some problems that seem to be caused by using all of the available threads and completely filling the queue. This is a portion of what we see in the log: _[action.search.type] [Kang the Conqueror] [779661445] Failed to execute fetch phase_ _org.elasticsearch.transport.RemoteTransportException: [Peter Noble][inet[/172.16.0.102:9300]][search/phase/fetch/id]_ _Caused by: org.elasticsearch.common.util.concurrent.EsRejectedExecutionException: rejected execution (queue capacity 1000) on org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler@10970469_ My question is, is it safe to increase the thread pool and queue size as a solution? Is it necessary to increase both? If my thoughts are correct, increasing the size of the thread pool should increase the amount of concurrent searches, which might eliminate the need for a larger queue? There is plenty of CPU available on all of the machines when this happens. Current settings are the default; which in our case is 72 for thread pool size and 1000 queue size. Does anyone have any suggestions on tweaking these settings? Any best practices to following? If this is not a proper solution what might be some alternatives? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/8f106018-0892-4dc2-a25a-b0722d6ae94c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/8f106018-0892-4dc2-a25a-b0722d6ae94c%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [May 19, 2014, 5:46pm UTC](https://discuss.elastic.co/t/increasing-thread-pool-queue-size/17579/2 "2014-05-19T17:46:20Z")
If search requests are being queued, then that means you probably do not have capacity for more concurrent searches. We are so many searches being queued? A temporary spike in search requests or are some expensive queries using up the existing threads? Try increasing the queue size and monitoring with is the maximum achieved. Cheers, Ivan On Sun, May 18, 2014 at 9:59 AM, [jcm112388@gmail.com](mailto:jcm112388@gmail.com) wrote: > We're running v1.1.0 in production right now. We have 15 nodes and 15 > shards w/ 4 replicas (75 total). > > Recently we've ran into some problems that seem to be caused by using all > of the available threads and completely filling the queue. This is a > portion of what we see in the log: > > _[action.search.type] [Kang the Conqueror] [779661445] Failed to execute > fetch phase_ > > _org.elasticsearch.transport.RemoteTransportException: [Peter > Noble][inet[/172.16.0.102:9300]][search/phase/fetch/id]_ > > _Caused by: > org.elasticsearch.common.util.concurrent.EsRejectedExecutionException: > rejected execution (queue capacity 1000) on > org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler@10970469_ > > My question is, is it safe to increase the thread pool and queue size as a > solution? Is it necessary to increase both? If my thoughts are correct, > increasing the size of the thread pool should increase the amount of > concurrent searches, which might eliminate the need for a larger queue? > > There is plenty of CPU available on all of the machines when this happens. > Current settings are the default; which in our case is 72 for thread pool > size and 1000 queue size. > > Does anyone have any suggestions on tweaking these settings? Any best > practices to following? > > If this is not a proper solution what might be some alternatives? > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/8f106018-0892-4dc2-a25a-b0722d6ae94c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/8f106018-0892-4dc2-a25a-b0722d6ae94c%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/8f106018-0892-4dc2-a25a-b0722d6ae94c%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/8f106018-0892-4dc2-a25a-b0722d6ae94c%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBKLs8JP-2w\_5tQoc9gX6sYS%2B6hM2umxi%2BYvu\_X\_w1umg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBKLs8JP-2w_5tQoc9gX6sYS%2B6hM2umxi%2BYvu_X_w1umg%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Greg\_Murnane](https://avatars.discourse-cdn.com/v4/letter/g/b9bd4f/32.png) [@Greg\_Murnane](https://discuss.elastic.co/u/Greg_Murnane)\ **Post date:** [May 19, 2014, 6:30pm UTC](https://discuss.elastic.co/t/increasing-thread-pool-queue-size/17579/3 "2014-05-19T18:30:48Z")
You might also keep an eye on what your disk utilization is like when the search queue is filling up; CPU isn't the only possible bottleneck here. -- The information transmitted in this email is intended only for the person(s) or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this email in error, please contact the sender and permanently delete the email from any computer. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/550eb225-95f1-438b-b3d5-b9996acc51b3%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/550eb225-95f1-438b-b3d5-b9996acc51b3%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:28am UTC](https://discuss.elastic.co/t/increasing-thread-pool-queue-size/17579/4 "2017-07-06T01:28:24Z")