How to settings threadpool on elastic cloud?

Elastic Stack Elasticsearch
1

My project using elastic cloud but reject error:
EsThreadPoolExecutor[name = instance-0000000003/write, queue capacity = 200

On breaking change description "Thread pool settings are now node-level settings. As such, it is not possible to update thread pool settings via the cluster settings API."
link here: https://www.elastic.co/guide/en/elasticsearch/reference/5.1/breaking_50_settings_changes.html#_forbid_changing_of_thread_pool_types

So, how to setting queue size on my elastic cloud?
Thanks.

1 Like
2

What is the full error you are seeing? Why do you think you need to change this setting? Are you by any chance indexing into a large number of indices and shards concurrently and as a result suffer from this?

3

Hi Christian,
Thanks for reply.

This is full error:
{
"error": {
"root_cause": [
{
"type": "es_rejected_execution_exception",
"reason": "rejected execution of processing of [193927725][indices:data/write/update[s]]: update {[comments][_doc][...], doc_as_upsert[true], doc[index {[null][_doc][null], source[...]}], scripted_upsert[false], detect_noop[true]} on EsThreadPoolExecutor[name = instance-0000000003/write, queue capacity = 200, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@788f8d44[Running, pool size = 16, active threads = 16, queued tasks = 200, completed tasks = 70440089]]"
}
],
"type": "es_rejected_execution_exception",
"reason": "rejected execution of processing of [193927725][indices:data/write/update[s]]: update {[comments][_doc][...], doc_as_upsert[true], doc[index {[null][_doc][null], source[...]}], scripted_upsert[false], detect_noop[true]} on EsThreadPoolExecutor[name = instance-0000000003/write, queue capacity = 200, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@788f8d44[Running, pool size = 16, active threads = 16, queued tasks = 200, completed tasks = 70440089]]"
},
"status": 429
}

My app listens to data from Facebook, therefore depends on traffic from Facebook.
My app usually has to write a lot of data at once from FB.

4

How many indices and shards are you actively indexing into? What is the size of your bulk requests?

5

This is the current my system info:
Nodes: 2
Indices: 66
Primary Shards: 150
Documents: 497m

App reveive data from FB and write by API /_updates, not using bulk request.

6

How many concurrent clients/threads do you have adding or modifying data in Elasticsearch?

Note that updating or indexing documents individually rather than in bulk will add a lot of overhead and result in lower performance compared to using bulk requests.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.