# Thread pool configuration

**URL:** <https://discuss.elastic.co/t/thread-pool-configuration/215683>\
**Category:** Elasticsearch\
**Created:** [January 20, 2020, 8:00am UTC](https://discuss.elastic.co/t/thread-pool-configuration/215683 "2020-01-20T08:00:20Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![Abhishek\_Tiwari](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abhishek_tiwari/32/45475_2.png) [@Abhishek\_Tiwari](https://discuss.elastic.co/u/Abhishek_Tiwari)\
**Post date:** [January 20, 2020, 8:00am UTC](https://discuss.elastic.co/t/thread-pool-configuration/215683/1 "2020-01-20T08:00:20Z")

</div>

Hi ,  
I am new in elastic search . here i want to know that how one should configure thread pool.

My current configuration is following

thread\_pool:

search:

size: 64

```
queue_size: 500

```

min\_queue\_size: 10  
max\_queue\_size: 1000  
auto\_queue\_frame\_size: 2000  
target\_response\_time: 1s  
http.enabled: true  
I am getting rejection on it .  
so what would be the best way configure it .

Regrads ,  
Abhishek

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [January 20, 2020, 8:21am UTC](https://discuss.elastic.co/t/thread-pool-configuration/215683/2 "2020-01-20T08:21:40Z")

</div>

> [@Abhishek\_Tiwari](#):
>
> so what would be the best way configure it .

The defaults are recommended for almost all cases. If you are getting rejections then your system is likely not powerful enough for your workload.

---

<div class="post-metadata">

**Author:** ![Abhishek\_Tiwari](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abhishek_tiwari/32/45475_2.png) [@Abhishek\_Tiwari](https://discuss.elastic.co/u/Abhishek_Tiwari)\
**Post date:** [January 20, 2020, 9:22am UTC](https://discuss.elastic.co/t/thread-pool-configuration/215683/3 "2020-01-20T09:22:41Z")

</div>

Thanks David,  
Then what should i do .  
Should i add more nodes ? or scaling up would work? .  
int((# of available\_processors \* 3) / 2) + 1 calculation for size would work for all kind of machines (Vm/ physical) ?.

---

<div class="post-metadata">

**Author:** ![Abhishek\_Tiwari](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abhishek_tiwari/32/45475_2.png) [@Abhishek\_Tiwari](https://discuss.elastic.co/u/Abhishek_Tiwari)\
**Post date:** [January 21, 2020, 5:11am UTC](https://discuss.elastic.co/t/thread-pool-configuration/215683/4 "2020-01-21T05:11:04Z")

</div>

@DavidTurner should i scaleout the cluster or scaling up will work ?

---

<div class="post-metadata">

**Author:** ![Abhishek\_Tiwari](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abhishek_tiwari/32/45475_2.png) [@Abhishek\_Tiwari](https://discuss.elastic.co/u/Abhishek_Tiwari)\
**Post date:** [January 22, 2020, 5:34am UTC](https://discuss.elastic.co/t/thread-pool-configuration/215683/5 "2020-01-22T05:34:40Z")

</div>

Hi All ,

Please help me here

---

<div class="post-metadata">

**Author:** ![DeeeFOX](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/deeefox/32/49566_2.png) [@DeeeFOX](https://discuss.elastic.co/u/DeeeFOX)\
**Post date:** [January 22, 2020, 6:25am UTC](https://discuss.elastic.co/t/thread-pool-configuration/215683/6 "2020-01-22T06:25:32Z")

</div>

Hi Abhishek, can u describe more detail here about when do u getting rejection on what operations to elasticsearch?

---

<div class="post-metadata">

**Author:** ![Abhishek\_Tiwari](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abhishek_tiwari/32/45475_2.png) [@Abhishek\_Tiwari](https://discuss.elastic.co/u/Abhishek_Tiwari)\
**Post date:** [January 22, 2020, 6:38am UTC](https://discuss.elastic.co/t/thread-pool-configuration/215683/7 "2020-01-22T06:38:40Z")

</div>

Thanks for replying .

cluster configuration  
Master -3 node (4 cpu, 20 gb ram)  
coordinating -3 node (16 cpu, 98 GB,)  
data- 5 node (28 cpu, 48 RAM)

And thread pool configuration is following  
thread\_pool:

search:

size: 64

```auto
queue_size: 500

```

min\_queue\_size: 10  
max\_queue\_size: 1000  
auto\_queue\_frame\_size: 2000  
target\_response\_time: 1s  
http.enabled: true

Here we are getting rejection on search type of queries .

How should i scale the cluster or can i optimize thread pool configuration .

Regards,  
Abhishek

---

<div class="post-metadata">

**Author:** ![DeeeFOX](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/deeefox/32/49566_2.png) [@DeeeFOX](https://discuss.elastic.co/u/DeeeFOX)\
**Post date:** [January 22, 2020, 6:47am UTC](https://discuss.elastic.co/t/thread-pool-configuration/215683/8 "2020-01-22T06:47:36Z")

</div>

> [@Abhishek\_Tiwari](#):
>
> queries

Are these queries heavy?  
(u can know that from the response information)

```
{
  "took" : 252,
  "timed_out" : false,
  "_shards" : {
    "total" : 1383, <------ here
    "successful" : 1383,
    "skipped" : 1368,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 2423, <------ here
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : []
  },
  ....
}

```

And can u give some example here?

And what kind of data are storing in ur cluster? what is the size of ur data, tb or pb?

---

<div class="post-metadata">

**Author:** ![Abhishek\_Tiwari](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abhishek_tiwari/32/45475_2.png) [@Abhishek\_Tiwari](https://discuss.elastic.co/u/Abhishek_Tiwari)\
**Post date:** [January 22, 2020, 6:57am UTC](https://discuss.elastic.co/t/thread-pool-configuration/215683/9 "2020-01-22T06:57:34Z")

</div>

We are indexing job data in ES to perform search . queries are mix both light weight and heavy. Index size on cluster is 1.5 TB (Total) .  
we are indexing the data in multiple index. search is being performed on 100 gb index. rest indexes are less in use. Traffic on site is 20k/min .

---

<div class="post-metadata">

**Author:** ![DeeeFOX](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/deeefox/32/49566_2.png) [@DeeeFOX](https://discuss.elastic.co/u/DeeeFOX)\
**Post date:** [January 22, 2020, 8:05am UTC](https://discuss.elastic.co/t/thread-pool-configuration/215683/10 "2020-01-22T08:05:10Z")

</div>

emmm, from information u give, i guess, thread pool settings is not the main reason...

May be the heavy queries and not well scaled data make this happen.

To the heavy queries more thing i want to know about are:  
1、Is this 100GB index static? Or still size increasing with new data writing on it？  
2、Is there a date type field in ur index, and all queries from user using this date type field to filter out the data?  
3、Will the queries search with "routing", "timeout" setting in request header？  
4、Will the queries search with "size", "from" setting in searching body？

To the not well scaled data more thing i want to know about are:  
1、how many shards are there in this 100GB index？  
2、Are data scaled well in every shard(doc counts and data size differ from each shard should not higher than 10%), and Are shards scaled well in every es data nodes(no more than 1 primary and 1 replica every node)？

ES's search perform is highly rely on  
1、disk performance  
2、off heap size  
3、suitable query body  
4、probably data scaling design

And please check these for ur use case and wish it will help u.

---

<div class="post-metadata">

**Author:** ![Abhishek\_Tiwari](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abhishek_tiwari/32/45475_2.png) [@Abhishek\_Tiwari](https://discuss.elastic.co/u/Abhishek_Tiwari)\
**Post date:** [January 30, 2020, 4:50am UTC](https://discuss.elastic.co/t/thread-pool-configuration/215683/11 "2020-01-30T04:50:45Z")

</div>

1、Is this 100GB index static? Or still size increasing with new data writing on it？  
[Abhishek]: it is dynamic . it is increasing but rate is slow .  
2、Is there a date type field in ur index, and all queries from user using this date type field to filter out the data?  
[Abhishek]: date field is present but we are not filtering on it.  
3、Will the queries search with "routing", "timeout" setting in request header？  
[Abhishek]: default. we are not using mine own routing .  
4、Will the queries search with "size", "from" setting in searching body？  
[Abhishek]: yes

To the not well scaled data more thing i want to know about are:  
1、how many shards are there in this 100GB index？  
[Abhishek] : 4  
2、Are data scaled well in every shard(doc counts and data size differ from each shard should not higher than 10%), and Are shards scaled well in every es data nodes(no more than 1 primary and 1 replica every node)？  
[Abhishek] . yes it is within limit .

---

<div class="post-metadata">

**Author:** ![Abhishek\_Tiwari](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abhishek_tiwari/32/45475_2.png) [@Abhishek\_Tiwari](https://discuss.elastic.co/u/Abhishek_Tiwari)\
**Post date:** [February 7, 2020, 10:04am UTC](https://discuss.elastic.co/t/thread-pool-configuration/215683/12 "2020-02-07T10:04:50Z")

</div>

@DeeeFOX please suggest

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 6, 2020, 10:04am UTC](https://discuss.elastic.co/t/thread-pool-configuration/215683/13 "2020-03-06T10:04:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.