# Increase queue size in AWS elasticsearch

**URL:** <https://discuss.elastic.co/t/increase-queue-size-in-aws-elasticsearch/105019>\
**Category:** Elasticsearch\
**Created:** [October 24, 2017, 8:56am UTC](https://discuss.elastic.co/t/increase-queue-size-in-aws-elasticsearch/105019 "2017-10-24T08:56:41Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![patricia022](https://avatars.discourse-cdn.com/v4/letter/p/7c8e57/32.png) [@patricia022](https://discuss.elastic.co/u/patricia022)\
**Post date:** [October 24, 2017, 8:56am UTC](https://discuss.elastic.co/t/increase-queue-size-in-aws-elasticsearch/105019/1 "2017-10-24T08:56:41Z")

</div>

I wasn't able to modify queue size in aws elasticsearch

I am getting this error  
{  
"Message": "Your request: '/\_cluster/settings' payload is not allowed."  
}

Elasticsearch query:  
PUT \_cluster/settings  
{  
"persistent" : {  
"threadpool.search.queue\_size" : 2000  
}

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [October 24, 2017, 9:07am UTC](https://discuss.elastic.co/t/increase-queue-size-in-aws-elasticsearch/105019/2 "2017-10-24T09:07:13Z")

</div>

AWS restricts access to a number of APIs and settings. We cannot help you with that sorry.

You would be better off using Elastic Cloud - [https://www.elastic.co/cloud/as-a-service](https://www.elastic.co/cloud/as-a-service)

---

<div class="post-metadata">

**Author:** ![patricia022](https://avatars.discourse-cdn.com/v4/letter/p/7c8e57/32.png) [@patricia022](https://discuss.elastic.co/u/patricia022)\
**Post date:** [October 24, 2017, 10:47am UTC](https://discuss.elastic.co/t/increase-queue-size-in-aws-elasticsearch/105019/3 "2017-10-24T10:47:00Z")

</div>

Is there any alternatives on how we can configure this?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 24, 2017, 6:06pm UTC](https://discuss.elastic.co/t/increase-queue-size-in-aws-elasticsearch/105019/4 "2017-10-24T18:06:32Z")

</div>

How come you need to change it in the first place? Are you having a very large number of concurrent search requests or just hitting a lot of shards with each request?

---

<div class="post-metadata">

**Author:** ![patricia022](https://avatars.discourse-cdn.com/v4/letter/p/7c8e57/32.png) [@patricia022](https://discuss.elastic.co/u/patricia022)\
**Post date:** [October 25, 2017, 6:24am UTC](https://discuss.elastic.co/t/increase-queue-size-in-aws-elasticsearch/105019/5 "2017-10-25T06:24:29Z")

</div>

Reason is I'm currently doing some load testing and I encountered this error

{"error":{"root\_cause":[{"type":"es\_rejected\_execution\_exception","reason":"rejected execution of org.elasticsearch.transport.TransportService$4@7868ca6f on EsThreadPoolExecutor[index, queue capacity = 200, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@7f817344[Running, pool size = 1, active threads = 1, queued tasks = 203, completed tasks = 668181]]"}],"type":"es\_rejected\_execution\_exception","reason":"rejected execution of org.elasticsearch.transport.TransportService$4@7868ca6f on EsThreadPoolExecutor[index, queue capacity = 200, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@7f817344[Running, pool size = 1, active threads = 1, queued tasks = 203, completed tasks = 668181]]"},"status":429}

Any thoughts on how I can fix this in AWS elasticsearch?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 25, 2017, 9:56am UTC](https://discuss.elastic.co/t/increase-queue-size-in-aws-elasticsearch/105019/6 "2017-10-25T09:56:51Z")

</div>

I do not think you can change that on AWS ES service.

Are you having a very large number of concurrent search requests or just hitting a lot of shards with each request?

---

<div class="post-metadata">

**Author:** ![patricia022](https://avatars.discourse-cdn.com/v4/letter/p/7c8e57/32.png) [@patricia022](https://discuss.elastic.co/u/patricia022)\
**Post date:** [October 26, 2017, 5:14am UTC](https://discuss.elastic.co/t/increase-queue-size-in-aws-elasticsearch/105019/7 "2017-10-26T05:14:54Z")

</div>

Yes I think its hitting a lot of shards with each request. I'm trying to simulate the scenario wherein I'm sending 1000 request for every one minute interval.

Is there any failover mechanism that I can use in order to handle this scenario? Any thoughts on where can I improve on or do?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 26, 2017, 8:19am UTC](https://discuss.elastic.co/t/increase-queue-size-in-aws-elasticsearch/105019/8 "2017-10-26T08:19:43Z")

</div>

How much data do you have? How large are your shards?

Make sure that you do not [too many small shards in your cluster](https://www.elastic.co/blog/how-many-shards-should-i-have-in-my-elasticsearch-cluster). You can determine the ideal shard size by following the procedure described in [this talk](https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing). This should allow you to fill up the queue less quickly.

---

<div class="post-metadata">

**Author:** ![patricia022](https://avatars.discourse-cdn.com/v4/letter/p/7c8e57/32.png) [@patricia022](https://discuss.elastic.co/u/patricia022)\
**Post date:** [October 26, 2017, 8:56am UTC](https://discuss.elastic.co/t/increase-queue-size-in-aws-elasticsearch/105019/9 "2017-10-26T08:56:26Z")

</div>

With regards to the data, I'm simulating 1 data per request and I'm triggering 1000 request per minute.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 23, 2017, 8:56am UTC](https://discuss.elastic.co/t/increase-queue-size-in-aws-elasticsearch/105019/10 "2017-11-23T08:56:31Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
