# Elasticsearch threadpool and index settings in ECK **URL:** \ **Category:** Elasticsearch\ **Created:** [January 15, 2021, 2:30pm UTC](https://discuss.elastic.co/t/elasticsearch-threadpool-and-index-settings-in-eck/261272 "2021-01-15T14:30:32Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![zohaib](https://avatars.discourse-cdn.com/v4/letter/z/3be4f8/32.png) [@zohaib](https://discuss.elastic.co/u/zohaib)\ **Post date:** [January 15, 2021, 2:30pm UTC](https://discuss.elastic.co/t/elasticsearch-threadpool-and-index-settings-in-eck/261272/1 "2021-01-15T14:30:32Z")
We are using ECK operator 1.2 and ElasticSearch 7.4.0 for a 3 node cluster with the default settings on Azure Kubernetes Services. We need to update the following ElasticSearch configuration in our cluster: threadpool.bulk.type: fixed threadpool.bulk.size: 24 threadpool.bulk.queue\_size: 1000 threadpool.search.type: fixed threadpool.search.size: 24 threadpool.search.queue\_size: 5 we have tried adding it under nodeSets.config: - name: default config: but elastic instance gets stuck on ApplyingChanges and elastic pods start crashing after that with the following error: "Suppressed: java.lang.IllegalArgumentException: unknown setting [node.threadpool.search.type] please check that any required plugins are installed, or check the breaking changes documentation for removed settings", What's the best method to make these changes for ElasticSearch cluster deployed using ECK on Kubernetes? Thanks in advance. ---
**Author:** ![angelo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/angelo/32/61325_2.png) [@angelo](https://discuss.elastic.co/u/angelo)\ **Post date:** [January 15, 2021, 8:08pm UTC](https://discuss.elastic.co/t/elasticsearch-threadpool-and-index-settings-in-eck/261272/2 "2021-01-15T20:08:57Z")
Further in the stack trace you should find more details, that often can guide you to the correct setting to use, for example: `Suppressed: java.lang.IllegalArgumentException: unknown setting [node.threadpool.search.queue_size] did you mean [thread_pool.search.queue_size]?` For your noted settings and referencing the 7.4 [docs](https://www.elastic.co/guide/en/elasticsearch/reference/7.4/modules-threadpool.html), it should only require: ```auto thread_pool.write.size: 24 thread_pool.write.queue_size: 1000 thread_pool.search.size: 24 thread_pool.search.queue_size: 50 ``` Note that many of the sizes are calculated off of the "# of available processors" that the node detects - this may then also require you to adjust the `processors` setting as noted in the documentation. I don't think we would generally recommend changing these settings, especially increasing the write/bulk thread pool if you are already encountering issues with bulk rejections anyway so please be aware of that. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [January 15, 2021, 8:27pm UTC](https://discuss.elastic.co/t/elasticsearch-threadpool-and-index-settings-in-eck/261272/3 "2021-01-15T20:27:47Z")
We definitely do not recommend increasing threadpool sizes, it just hides the underlying issue. [Any idea what these errors mean version 2.4.2](https://discuss.elastic.co/t/any-idea-what-these-errors-mean-version-2-4-2/70690/5) is an old but entirely relevant explanation as to why. ---
**Author:** ![zohaib](https://avatars.discourse-cdn.com/v4/letter/z/3be4f8/32.png) [@zohaib](https://discuss.elastic.co/u/zohaib)\ **Post date:** [January 15, 2021, 11:20pm UTC](https://discuss.elastic.co/t/elasticsearch-threadpool-and-index-settings-in-eck/261272/4 "2021-01-15T23:20:15Z")
Got it. We are trying to find the root cause of the below errors: invalid NEST response built from a unsuccessful () low level call on POST: /\_bulk?refresh=wait\_for # Invalid Bulk Items ---\> System.IO.IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request.. ---\> System.Net.Sockets.SocketException (995): The I/O operation has been aborted because of either a thread exit or an application request. What are your recommendations? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [January 17, 2021, 10:28pm UTC](https://discuss.elastic.co/t/elasticsearch-threadpool-and-index-settings-in-eck/261272/5 "2021-01-17T22:28:52Z")
What do your Elasticsearch logs show at the time of this error? ---
**Author:** ![janos](https://avatars.discourse-cdn.com/v4/letter/j/ad7895/32.png) [@janos](https://discuss.elastic.co/u/janos)\ **Post date:** [January 18, 2021, 9:53am UTC](https://discuss.elastic.co/t/elasticsearch-threadpool-and-index-settings-in-eck/261272/6 "2021-01-18T09:53:55Z")
Hello Guys, Thanks for your help. I will try to explain briefly what is happening so that you can react to whether our indexing strategy is the source of this issue causing overload of the Elastic indexing service or something else. The only strange thing is that it is working when we host the Elastic inside our Windows running \bin\elasticsearch.bat but inside POD is always failing. So, the current situation is that we have a bunch of documents with metadata. Each metadata field is a document in the index with a relation field to the parent (document). When we process these documents parallel, the followings happen: ``` Document 1 - Request 1: Index Document Head - Request 2: Index Document Metadata Fields (Bulk Request) - Request 3: Index Document Text Extract Document 2 - Request 1: Index Document Head - Request 2: Index Document Metadata Fields (Bulk Request) - Request 3: Index Document Text Extract ``` So, if we have 100 documents processing them in batches parallel, it can mean 100 / batch size requests (e.g. if batch size 4, then 25 requests) for writing the same index at the same time. Request 1, 2, 3 (which are different methods in the code as well) run sequentially awaiting each other so you can take them as ones. The question is whether it can be the reason of this issue and if yes, then we should implement a different indexing strategy working with larger and more composite batches per a request-base? As a second alternate solution, we could save documents and metadata to the persistent storage parallel but indexing would happen sequentially, in a dedicated thread taking available items from a queue continously because we want to make documents available for search immediately without having to wait for the last document to be saved. _Update_ It seems that the team has managed to fix this issue. The backround job calling the microservice endpoint to save and index documents frequently times out. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 15, 2021, 9:53am UTC](https://discuss.elastic.co/t/elasticsearch-threadpool-and-index-settings-in-eck/261272/7 "2021-02-15T09:53:56Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.