# Elasticsearch thread pool rejected

**URL:** <https://discuss.elastic.co/t/elasticsearch-thread-pool-rejected/223142>\
**Category:** Elasticsearch\
**Created:** [March 11, 2020, 1:41pm UTC](https://discuss.elastic.co/t/elasticsearch-thread-pool-rejected/223142 "2020-03-11T13:41:48Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Guylot](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guylot/32/47141_2.png) [@Guylot](https://discuss.elastic.co/u/Guylot)\
**Post date:** [March 11, 2020, 1:41pm UTC](https://discuss.elastic.co/t/elasticsearch-thread-pool-rejected/223142/1 "2020-03-11T13:41:49Z")

</div>

Hi,  
We are using Elasticsearch 5.6 (Next week we will upgrade to 7.6) for our application, we are using elasticsearch mostly for bulk indexing and much fewer for querying the data.  
Our cluster contains 8 nodes. 1 Data node and the others are both data and master eligible.  
Each node has 8G RAM (4G from it for heap usage) and 2CPU.  
We have 2 daily indexes, daily log index with two shards (Around 500k-1mil documents per day) and daily data index with two shards (Around 4-5 mil documents per day).  
Lately we get lot of rejected from BULK and SEARCH thread pools we tried to increase our nodes from 6 to 8 but we still see some rejected operations even it did reduce the rejected rate.  
Each node doesn't use high amount of CPU or heap but sometimes the load is going high.  
We don't know what is the best way to solve those errors or how to identified the cause.. Increase shard size/ stronger machines/ split to dedicated master and data nodes..

How can we fix the rejected?

Thanks

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [March 11, 2020, 10:47pm UTC](https://discuss.elastic.co/t/elasticsearch-thread-pool-rejected/223142/2 "2020-03-11T22:47:12Z")

</div>

How are you sending the data to Elasticsearch?

---

<div class="post-metadata">

**Author:** ![Guylot](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guylot/32/47141_2.png) [@Guylot](https://discuss.elastic.co/u/Guylot)\
**Post date:** [March 12, 2020, 7:39am UTC](https://discuss.elastic.co/t/elasticsearch-thread-pool-rejected/223142/3 "2020-03-12T07:39:08Z")

</div>

We are using C# Nest API Bulk method

---

<div class="post-metadata">

**Author:** ![rameshkr1994](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rameshkr1994/32/59029_2.png) [@rameshkr1994](https://discuss.elastic.co/u/rameshkr1994)\
**Post date:** [March 12, 2020, 7:51am UTC](https://discuss.elastic.co/t/elasticsearch-thread-pool-rejected/223142/4 "2020-03-12T07:51:48Z")

</div>

> [@Guylot](#):
>
> How can we fix the rejected?

`above error basically seen when data types or index mapping is wrong as compare to source index data to destination index data`

> i think you need , check either your data types or index mapping are correct or not?.

Thanks  
HadoopHelp

---

<div class="post-metadata">

**Author:** ![Guylot](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guylot/32/47141_2.png) [@Guylot](https://discuss.elastic.co/u/Guylot)\
**Post date:** [March 12, 2020, 8:18am UTC](https://discuss.elastic.co/t/elasticsearch-thread-pool-rejected/223142/5 "2020-03-12T08:18:44Z")

</div>

The mapping seems correct.  
Those are the errors we get from one logged bulk operation (the log contains more success and failed operations this are just 2 samples from the full log)

```
{
  "Errors": true,
  "IsValid": false,
  "Items": [
    {
      "Matches": null,
      "Operation": "index",
      "Error": null,
      "Id": "AXDMCn9LrIcxTwmun01C",
      "Index": "data_20200311",
      "IsValid": true,
      "PrimaryTerm": 0,
      "Result": "created",
      "SequenceNumber": 0,
      "Shards": {
        "Failed": 0,
        "Failures": [],
        "Successful": 2,
        "Total": 2
      },
      "Status": 201,
      "Type": "data",
      "Version": 1
    },
    {
      "Matches": null,
      "Operation": "index",
      "Error": {
        "Index": null,
        "Shard": 0,
        "Headers": {},
        "RootCause": null,
        "CausedBy": null,
        "Metadata": {
          "BytesLimit": null,
          "BytesWanted": null,
          "Column": null,
          "FailedShards": [],
          "Grouped": null,
          "Index": null,
          "IndexUUID": null,
          "Language": null,
          "LicensedExpiredFeature": null,
          "Line": null,
          "Phase": null,
          "ResourceId": [],
          "ResourceType": null,
          "Script": null,
          "ScriptStack": [],
          "Shard": null
        },
        "Reason": "rejected execution of org.elasticsearch.transport.TransportService$7@4a5feb40 on EsThreadPoolExecutor[bulk, queue capacity = 200, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@723c2a72[Running, pool size = 2, active threads = 2, queued tasks = 200, completed tasks = 77533164]]",
        "StackTrace": null,
        "Type": "es_rejected_execution_exception"
      },
      "Id": "AXDMCn9LrIcxTwmun01D",
      "Index": "data_20200311",
      "IsValid": false,
      "PrimaryTerm": 0,
      "Result": null,
      "SequenceNumber": 0,
      "Shards": null,
      "Status": 429,
      "Type": "data",
      "Version": 0
    }
  ]
}

```

From the log its look like some operations success and some fail for full bulk thread pool

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [March 12, 2020, 9:32pm UTC](https://discuss.elastic.co/t/elasticsearch-thread-pool-rejected/223142/6 "2020-03-12T21:32:38Z")

</div>

> [@rameshkr1994](#):
>
> above error basically seen when data types or index mapping is wrong as compare to source index data to destination index data

No, it has nothing to do with it.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 9, 2020, 9:32pm UTC](https://discuss.elastic.co/t/elasticsearch-thread-pool-rejected/223142/7 "2020-04-09T21:32:41Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.